[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:20:48 dist-test-slave-1vmg krb5kdc[9592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:20:48 dist-test-slave-1vmg krb5kdc[9592](info): set up 2 sockets
Aug 09 19:20:48 dist-test-slave-1vmg krb5kdc[9592](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 19:20:49.977056 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:20:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:20:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:20:50.086656 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43477
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:43477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:50.171614 9610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:50.171797 9610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:50.171818 9610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:50.172886 9610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:20:50.172919 9610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:50.172931 9610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:50.172942 9610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:20:50.172953 9610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:20:50.174314 9610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:43477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:50.174489 9610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:50.174734 9610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:50.180810 9615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:50.180863 9616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:50.181196 9620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:50.181306 9610 server_base.cc:1034] running on GCE node
I20260809 19:20:50.181540 9610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:50.181840 9610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:50.182974 9610 hybrid_clock.cc:648] HybridClock initialized: now 1786303250182972 us; error 63 us; skew 500 ppm
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:50.185313 9610 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:20:50.185758 9610 webserver.cc:460] Webserver started at http://127.9.91.126:35745/ using document root <none> and password file <none>
I20260809 19:20:50.185945 9610 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:50.185992 9610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:50.186121 9610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:50.187450 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "1a33faeeb13248eab696823e81a60d19"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "a0525a16c7b40cf99b460872b2b801f9"
server_key_iv: "3a2416d14385e8ea73a91b54569b0d04"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.188206 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "1a33faeeb13248eab696823e81a60d19"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "a0525a16c7b40cf99b460872b2b801f9"
server_key_iv: "3a2416d14385e8ea73a91b54569b0d04"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.189685 9610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:20:50.190372 9631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.190524 9610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:50.190610 9610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "1a33faeeb13248eab696823e81a60d19"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "a0525a16c7b40cf99b460872b2b801f9"
server_key_iv: "3a2416d14385e8ea73a91b54569b0d04"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.190668 9610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:50.210384 9610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:50.212224 9610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:50.212347 9610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:50.216679 9610 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:43477
I20260809 19:20:50.216717 9714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:43477 every 8 connection(s)
I20260809 19:20:50.216997 9610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:20:50.217844 9715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:50.220302 9715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19: Bootstrap starting.
I20260809 19:20:50.220702 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9610
I20260809 19:20:50.220788 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:20:50.220867 9715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:50.221328 9715 log.cc:824] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:50.221385 9581 external_mini_cluster.cc:1428] Setting key 8a78703ced9e26d3b16c225898922bd3
I20260809 19:20:50.222086 9715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19: No bootstrap required, opened a new log
I20260809 19:20:50.226246 9715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a33faeeb13248eab696823e81a60d19" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43477 } }
I20260809 19:20:50.226421 9715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:50.226464 9715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a33faeeb13248eab696823e81a60d19, State: Initialized, Role: FOLLOWER
I20260809 19:20:50.226619 9715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a33faeeb13248eab696823e81a60d19" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43477 } }
I20260809 19:20:50.226711 9715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:20:50.226743 9715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:20:50.226792 9715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:50.227907 9715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a33faeeb13248eab696823e81a60d19" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43477 } }
I20260809 19:20:50.228004 9715 leader_election.cc:302] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a33faeeb13248eab696823e81a60d19; no voters:
I20260809 19:20:50.228153 9715 leader_election.cc:288] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:20:50.228199 9722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:50.228307 9722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [term 1 LEADER]: Becoming Leader. State: Replica: 1a33faeeb13248eab696823e81a60d19, State: Running, Role: LEADER
I20260809 19:20:50.228384 9722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a33faeeb13248eab696823e81a60d19" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43477 } }
I20260809 19:20:50.228389 9715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:20:50.229244 9726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a33faeeb13248eab696823e81a60d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a33faeeb13248eab696823e81a60d19" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43477 } } }
I20260809 19:20:50.229497 9726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:50.230218 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a33faeeb13248eab696823e81a60d19. Latest consensus state: current_term: 1 leader_uuid: "1a33faeeb13248eab696823e81a60d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a33faeeb13248eab696823e81a60d19" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43477 } } }
I20260809 19:20:50.230295 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19 [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:50.230631 9734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:20:50.231583 9734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:20:50.234179 9734 catalog_manager.cc:1347] Generated new cluster ID: d3d60d792bed44a18aebdf74f30cc923
I20260809 19:20:50.234236 9734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:50.240016 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.222575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.9.91.126:43477)
0809 19:20:50.222837 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:50.222847 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:50.223082 (+ 235us) server_negotiation.cc:407] Connection header received
0809 19:20:50.226229 (+ 3147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:50.226236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:50.226350 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:50.226470 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:50.228741 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.230220 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.231946 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.232540 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.232650 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.236556 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:50.236566 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:50.236566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:50.236576 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:50.238240 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.238611 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.238614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.238614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.238723 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.238970 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.238972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.238972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.239187 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:50.239409 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:50.239612 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 19:20:50.239792 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260809 19:20:50.253393 9734 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:20:50.254050 9734 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:20:50.264374 9734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a33faeeb13248eab696823e81a60d19: Generated new TSK 0
I20260809 19:20:50.264523 9734 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:20:50.293355 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43477
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:50.391745 9755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:50.391933 9755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:50.391955 9755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:50.393038 9755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:50.393076 9755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:50.393121 9755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:20:50.394587 9755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43477
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:50.394750 9755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:50.394909 9755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:50.399629 9765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:50.399659 9760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:50.399664 9763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:50.400175 9755 server_base.cc:1034] running on GCE node
I20260809 19:20:50.400354 9755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:50.400547 9755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:50.401684 9755 hybrid_clock.cc:648] HybridClock initialized: now 1786303250401675 us; error 26 us; skew 500 ppm
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:50.404084 9755 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:20:50.404476 9755 webserver.cc:460] Webserver started at http://127.9.91.65:37247/ using document root <none> and password file <none>
I20260809 19:20:50.404667 9755 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:50.404717 9755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:50.404847 9755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:50.405889 9755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "ebfa1835c63f4c5fbcac02b39f6afefe"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "5ec59add3d65428f448531303cb26413"
server_key_iv: "b7486c14ddd09a071d7b39d65dc64c37"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.406311 9755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "ebfa1835c63f4c5fbcac02b39f6afefe"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "5ec59add3d65428f448531303cb26413"
server_key_iv: "b7486c14ddd09a071d7b39d65dc64c37"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.407727 9755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:20:50.408399 9773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.408547 9755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.408633 9755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "ebfa1835c63f4c5fbcac02b39f6afefe"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "5ec59add3d65428f448531303cb26413"
server_key_iv: "b7486c14ddd09a071d7b39d65dc64c37"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.408695 9755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:50.426697 9755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:50.428483 9755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:50.428612 9755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:50.428794 9755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:50.429055 9755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:50.429090 9755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.429116 9755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:50.429132 9755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.436160 9755 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:33701
I20260809 19:20:50.436195 9943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:33701 every 8 connection(s)
I20260809 19:20:50.436455 9755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:20:50.437825 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9755
I20260809 19:20:50.437918 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:20:50.438063 9581 external_mini_cluster.cc:1428] Setting key 74efb0f7174f68a56eaf1b1a16984e39
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:50.446033 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.437121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:60153 (local address 127.9.91.126:43477)
0809 19:20:50.437178 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:50.437180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:50.437379 (+ 199us) server_negotiation.cc:407] Connection header received
0809 19:20:50.437881 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:50.437883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:50.437898 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:50.437965 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:50.439127 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.439787 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.441003 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.441269 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.441324 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.443311 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:50.443312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:50.443313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:50.443320 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:50.444791 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.445186 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.445187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.445187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.445249 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.445447 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.445448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.445448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.445577 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:50.445649 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:50.445849 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 19:20:50.445929 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:20:50.446050 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.437100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43477 (local address 127.9.91.65:60153)
0809 19:20:50.437259 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:50.437295 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:50.437784 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:50.438030 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:50.438034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:50.438829 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:50.439045 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:50.439051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.439862 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.439862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:50.440912 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:50.440916 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.441351 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.441352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:50.441456 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.441866 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:50.441874 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:50.443212 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:50.444876 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:50.444883 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:50.444885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:50.445114 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:50.445313 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:50.445314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:50.445314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:50.445396 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:50.445635 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:50.445638 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:50.445736 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 19:20:50.445807 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:20:50.445911 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260809 19:20:50.446585 9944 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43477
I20260809 19:20:50.446682 9944 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:50.446848 9944 heartbeater.cc:511] Master 127.9.91.126:43477 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
I20260809 19:20:50.447391 9661 ts_manager.cc:194] Registered new tserver with Master: ebfa1835c63f4c5fbcac02b39f6afefe (127.9.91.65:33701)
I20260809 19:20:50.448100 9661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:60153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:20:50.461727 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43477
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:50.543831 9953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:50.544023 9953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:50.544049 9953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:50.545116 9953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:50.545152 9953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:50.545179 9953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:20:50.546562 9953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43477
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:50.546744 9953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:50.546944 9953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:50.551604 9960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:50.551779 9953 server_base.cc:1034] running on GCE node
W20260809 19:20:50.551604 9961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:50.551585 9963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:50.552057 9953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:50.552282 9953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:50.553422 9953 hybrid_clock.cc:648] HybridClock initialized: now 1786303250553411 us; error 30 us; skew 500 ppm
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:50.555877 9953 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:20:50.556345 9953 webserver.cc:460] Webserver started at http://127.9.91.66:45863/ using document root <none> and password file <none>
I20260809 19:20:50.556567 9953 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:50.556615 9953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:50.556741 9953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:50.557729 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "5641773d06914516bc03d7d9a165f3fd"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "03ce778932a399ac25480f16b3d25003"
server_key_iv: "7d367abcfdb5d647cd439abd7624d247"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.558089 9953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "5641773d06914516bc03d7d9a165f3fd"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "03ce778932a399ac25480f16b3d25003"
server_key_iv: "7d367abcfdb5d647cd439abd7624d247"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.559396 9953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:20:50.560101 9971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.560249 9953 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.560330 9953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "5641773d06914516bc03d7d9a165f3fd"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "03ce778932a399ac25480f16b3d25003"
server_key_iv: "7d367abcfdb5d647cd439abd7624d247"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.560386 9953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:50.572525 9953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:50.574082 9953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:50.574208 9953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:50.574421 9953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:50.574687 9953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:50.574720 9953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.574746 9953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:50.574772 9953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.584139 9953 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:41641
I20260809 19:20:50.584174 10135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:41641 every 8 connection(s)
I20260809 19:20:50.584491 9953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:20:50.586193 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9953
I20260809 19:20:50.586301 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:20:50.586449 9581 external_mini_cluster.cc:1428] Setting key 29e45da31889b3860f62253c99f87a29
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:50.594017 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.585121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:37293 (local address 127.9.91.126:43477)
0809 19:20:50.585170 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:50.585172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:50.585451 (+ 279us) server_negotiation.cc:407] Connection header received
0809 19:20:50.585879 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:50.585880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:50.585894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:50.585945 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:50.586954 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.587622 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.588549 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.588823 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.588899 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.591231 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:50.591233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:50.591234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:50.591243 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:50.592711 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.593150 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.593151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.593151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.593209 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.593426 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.593427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.593428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.593539 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:50.593593 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:50.593838 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 19:20:50.593919 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:20:50.594144 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.585105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43477 (local address 127.9.91.66:37293)
0809 19:20:50.585342 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:50.585372 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:50.585789 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:50.586005 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:50.586009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:50.586603 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:50.586850 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:50.586856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.587705 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.587706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:50.588461 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:50.588463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.588906 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.588907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:50.589012 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.589493 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:50.589503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:50.591147 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:50.592786 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:50.592794 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:50.592796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:50.593081 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:50.593262 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:50.593263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:50.593264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:50.593369 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:50.593595 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:50.593602 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:50.593727 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 19:20:50.593811 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:20:50.593954 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":167,"threads_started":1}
I20260809 19:20:50.594645 10137 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43477
I20260809 19:20:50.594725 10137 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:50.594880 10137 heartbeater.cc:511] Master 127.9.91.126:43477 requested a full tablet report, sending...
I20260809 19:20:50.595271 9661 ts_manager.cc:194] Registered new tserver with Master: 5641773d06914516bc03d7d9a165f3fd (127.9.91.66:41641)
I20260809 19:20:50.595597 9661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:37293
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:20:50.622393 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43477
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:50.704931 10147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:50.705120 10147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:50.705156 10147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:50.706254 10147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:50.706300 10147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:50.706339 10147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:20:50.707780 10147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43477
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:50.708002 10147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:50.708206 10147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:50.712886 10158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:50.713164 10147 server_base.cc:1034] running on GCE node
W20260809 19:20:50.712872 10153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:50.712927 10152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:50.713449 10147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:50.713721 10147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:50.714875 10147 hybrid_clock.cc:648] HybridClock initialized: now 1786303250714858 us; error 28 us; skew 500 ppm
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:50.717286 10147 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:20:50.717711 10147 webserver.cc:460] Webserver started at http://127.9.91.67:40991/ using document root <none> and password file <none>
I20260809 19:20:50.717926 10147 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:50.717975 10147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:50.718115 10147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:50.719203 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "d588ecac7fd34ee0bf049b9a7c0b2e0d"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "d1014709557c232ffce537060cba5eec"
server_key_iv: "6a97beb8bf4d72242ab0fc5aee01b8f1"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.719601 10147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "d588ecac7fd34ee0bf049b9a7c0b2e0d"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "d1014709557c232ffce537060cba5eec"
server_key_iv: "6a97beb8bf4d72242ab0fc5aee01b8f1"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.721057 10147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:50.721709 10167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.721858 10147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:50.721932 10147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "d588ecac7fd34ee0bf049b9a7c0b2e0d"
format_stamp: "Formatted at 2026-08-09 19:20:50 on dist-test-slave-1vmg"
server_key: "d1014709557c232ffce537060cba5eec"
server_key_iv: "6a97beb8bf4d72242ab0fc5aee01b8f1"
server_key_version: "encryptionkey@0"
I20260809 19:20:50.722012 10147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:50.740633 10147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:50.742224 10147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:50.742358 10147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:50.742616 10147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:50.743048 10147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:50.743091 10147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.743124 10147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:50.743150 10147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:50.753003 10147 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:43675
I20260809 19:20:50.753032 10342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:43675 every 8 connection(s)
I20260809 19:20:50.753336 10147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:20:50.757230 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10147
I20260809 19:20:50.757313 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:20:50.757447 9581 external_mini_cluster.cc:1428] Setting key fb2b6d237f560905d6cf1d2c269074c6
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:50.762065 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.754034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:42901 (local address 127.9.91.126:43477)
0809 19:20:50.754107 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:50.754109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:50.754274 (+ 165us) server_negotiation.cc:407] Connection header received
0809 19:20:50.754770 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:50.754771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:50.754781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:50.754826 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:50.755766 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.756255 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.757242 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.757535 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.757613 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.759952 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:50.759953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:50.759953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:50.759960 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:50.761030 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.761381 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.761381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.761382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.761427 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.761601 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.761602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.761602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.761696 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:50.761740 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:50.761929 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:20:50.761990 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:20:50.762184 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.754001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43477 (local address 127.9.91.67:42901)
0809 19:20:50.754164 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:50.754199 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:50.754688 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:50.754884 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:50.754887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:50.755509 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:50.755694 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:50.755699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.756353 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.756354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:50.757144 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:50.757147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.757621 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.757622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:50.757722 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.758293 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:50.758302 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:50.759866 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:50.761117 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:50.761124 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:50.761126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:50.761327 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:50.761469 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:50.761470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:50.761470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:50.761554 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:50.761736 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:50.761739 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:50.761840 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 19:20:50.761910 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:20:50.762019 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260809 19:20:50.762604 10346 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43477
I20260809 19:20:50.762714 10346 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:50.762877 10346 heartbeater.cc:511] Master 127.9.91.126:43477 requested a full tablet report, sending...
I20260809 19:20:50.763242 9661 ts_manager.cc:194] Registered new tserver with Master: d588ecac7fd34ee0bf049b9a7c0b2e0d (127.9.91.67:43675)
I20260809 19:20:50.763597 9661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:42901
I20260809 19:20:50.769203 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:20:50.791735 10354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.785136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.9.91.65:33701)
0809 19:20:50.785299 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:50.785303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:50.785312 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:20:50.785335 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:50.785336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:50.785348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:50.785412 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:50.785869 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.786593 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.787296 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.787673 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.787729 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.789627 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:50.789635 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:50.789635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:50.789641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:50.790844 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.791155 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.791157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.791157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.791209 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.791333 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.791333 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.791333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.791441 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:50.791532 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:50.791582 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 19:20:50.791643 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
W20260809 19:20:50.792071 9815 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41804
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[9592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:20:50.812826 10354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:50.807141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.9.91.65:33701)
0809 19:20:50.807190 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:50.807192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:50.807255 (+ 63us) server_negotiation.cc:407] Connection header received
0809 19:20:50.807317 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:50.807318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:50.807329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:50.807374 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:50.807868 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.808322 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.808983 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:50.809183 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:50.809235 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:50.811077 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:50.811078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:50.811079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:50.811084 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:50.812079 (+ 995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.812338 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.812339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.812339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.812392 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:50.812510 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:50.812511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:50.812511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:50.812586 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:50.812662 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:50.812715 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 19:20:50.812771 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:20:50.813233 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9755
I20260809 19:20:50.829015 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9953
I20260809 19:20:50.844373 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10147
I20260809 19:20:50.852365 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9610
2026-08-09T19:20:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2243 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[10364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[10364](info): set up 2 sockets
Aug 09 19:20:50 dist-test-slave-1vmg krb5kdc[10364](info): commencing operation
krb5kdc: starting...
W20260809 19:20:51.983214 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:20:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:20:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:20:52.097241 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35685
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:35685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:52.177240 10382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:52.177419 10382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:52.177440 10382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:52.178479 10382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:20:52.178509 10382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:52.178522 10382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:52.178534 10382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:20:52.178550 10382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:20:52.179991 10382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:35685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:52.180186 10382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:52.180349 10382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:52.184798 10390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:52.184818 10389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:52.184856 10392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:52.184849 10382 server_base.cc:1034] running on GCE node
I20260809 19:20:52.185211 10382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:52.185462 10382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:52.186587 10382 hybrid_clock.cc:648] HybridClock initialized: now 1786303252186578 us; error 28 us; skew 500 ppm
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:52.188930 10382 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:20:52.189338 10382 webserver.cc:460] Webserver started at http://127.9.91.126:43213/ using document root <none> and password file <none>
I20260809 19:20:52.189548 10382 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:52.189592 10382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:52.189713 10382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:52.190680 10382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "546bb7f7d5ec463fb49514cd653c736f"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "010cb2f29a2936886dacbd20b9f6c64b"
server_key_iv: "9d3303d1f0c944abe6073489bb3a550d"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.191035 10382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "546bb7f7d5ec463fb49514cd653c736f"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "010cb2f29a2936886dacbd20b9f6c64b"
server_key_iv: "9d3303d1f0c944abe6073489bb3a550d"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.192303 10382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:20:52.193022 10402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.193177 10382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:20:52.193267 10382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "546bb7f7d5ec463fb49514cd653c736f"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "010cb2f29a2936886dacbd20b9f6c64b"
server_key_iv: "9d3303d1f0c944abe6073489bb3a550d"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.193341 10382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:52.206458 10382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:52.208488 10382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:52.208627 10382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:52.212527 10382 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:35685
I20260809 19:20:52.212551 10484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:35685 every 8 connection(s)
I20260809 19:20:52.212850 10382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:20:52.213706 10487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:52.216112 10487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f: Bootstrap starting.
I20260809 19:20:52.216670 10487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:52.216912 10487 log.cc:824] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:52.217598 10487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f: No bootstrap required, opened a new log
I20260809 19:20:52.219261 10487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546bb7f7d5ec463fb49514cd653c736f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35685 } }
I20260809 19:20:52.219439 10487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:52.219499 10487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 546bb7f7d5ec463fb49514cd653c736f, State: Initialized, Role: FOLLOWER
I20260809 19:20:52.219619 10487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546bb7f7d5ec463fb49514cd653c736f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35685 } }
I20260809 19:20:52.219677 10487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:20:52.219703 10487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:20:52.219751 10487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:52.220263 10487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546bb7f7d5ec463fb49514cd653c736f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35685 } }
I20260809 19:20:52.220355 10487 leader_election.cc:302] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 546bb7f7d5ec463fb49514cd653c736f; no voters:
I20260809 19:20:52.220480 10487 leader_election.cc:288] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:20:52.220542 10490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:52.220706 10490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [term 1 LEADER]: Becoming Leader. State: Replica: 546bb7f7d5ec463fb49514cd653c736f, State: Running, Role: LEADER
I20260809 19:20:52.220727 10487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:20:52.220803 10490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546bb7f7d5ec463fb49514cd653c736f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35685 } }
I20260809 19:20:52.221174 10495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 546bb7f7d5ec463fb49514cd653c736f. Latest consensus state: current_term: 1 leader_uuid: "546bb7f7d5ec463fb49514cd653c736f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546bb7f7d5ec463fb49514cd653c736f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35685 } } }
I20260809 19:20:52.221283 10495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:52.221387 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10382
I20260809 19:20:52.221482 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:20:52.221587 10494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "546bb7f7d5ec463fb49514cd653c736f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "546bb7f7d5ec463fb49514cd653c736f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35685 } } }
I20260809 19:20:52.221671 9581 external_mini_cluster.cc:1428] Setting key 2b2698d8b0031ca24786970a93dcec61
I20260809 19:20:52.221711 10494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:52.221655 10504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:20:52.222287 10504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:20:52.224200 10504 catalog_manager.cc:1347] Generated new cluster ID: 563989ece1474daa815e21ebd40c6033
I20260809 19:20:52.224254 10504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:52.235000 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.222470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.9.91.126:35685)
0809 19:20:52.222660 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.222670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.223401 (+ 731us) server_negotiation.cc:407] Connection header received
0809 19:20:52.223511 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.223518 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.223585 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.223709 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.227404 (+ 3695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.228309 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.229051 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.229452 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.229565 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.231832 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.231836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.231836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.231847 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.233444 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.233803 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.233806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.233806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.233894 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.234094 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.234095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.234095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.234322 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.234474 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.234603 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.234740 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260809 19:20:52.243503 10504 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:20:52.243961 10504 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:20:52.248137 10504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 546bb7f7d5ec463fb49514cd653c736f: Generated new TSK 0
I20260809 19:20:52.248287 10504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:20:52.279049 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35685
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:52.390512 10533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:52.390689 10533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:52.390712 10533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:52.391829 10533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:52.391868 10533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:52.391893 10533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:20:52.393321 10533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:52.393498 10533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:52.393673 10533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:52.398257 10545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:52.398306 10548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:52.398247 10546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:52.398552 10533 server_base.cc:1034] running on GCE node
I20260809 19:20:52.398747 10533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:52.399004 10533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:52.400158 10533 hybrid_clock.cc:648] HybridClock initialized: now 1786303252400144 us; error 34 us; skew 500 ppm
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:52.402513 10533 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:20:52.403012 10533 webserver.cc:460] Webserver started at http://127.9.91.65:37475/ using document root <none> and password file <none>
I20260809 19:20:52.403237 10533 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:52.403283 10533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:52.403409 10533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:52.404408 10533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "5524f691bc234bb88eb8e70acdec2842"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "eea0a5f79d2f80aff0582fe3681b7d3c"
server_key_iv: "0b8660367468c4e665f6577a200f60b2"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.404732 10533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "5524f691bc234bb88eb8e70acdec2842"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "eea0a5f79d2f80aff0582fe3681b7d3c"
server_key_iv: "0b8660367468c4e665f6577a200f60b2"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.405916 10533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:20:52.406548 10559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.406708 10533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:52.406790 10533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "5524f691bc234bb88eb8e70acdec2842"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "eea0a5f79d2f80aff0582fe3681b7d3c"
server_key_iv: "0b8660367468c4e665f6577a200f60b2"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.406853 10533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:52.421461 10533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:52.423292 10533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:52.423416 10533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:52.423606 10533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:52.423875 10533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:52.423908 10533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.423934 10533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:52.423951 10533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.429859 10533 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:44631
I20260809 19:20:52.429898 10720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:44631 every 8 connection(s)
I20260809 19:20:52.430181 10533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:52.437408 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10533
I20260809 19:20:52.437522 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:20:52.437695 9581 external_mini_cluster.cc:1428] Setting key c48a8fddb705aa85da7205c942315716
I20260809 19:20:52.439496 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.430904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:48099 (local address 127.9.91.126:35685)
0809 19:20:52.430984 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.430986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.431172 (+ 186us) server_negotiation.cc:407] Connection header received
0809 19:20:52.432212 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.432213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.432224 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.432273 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.432877 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.433391 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.434385 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.434590 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.434641 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.436670 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.436672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.436672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.436680 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.438155 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.438598 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.438600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.438600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.438660 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.438875 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.438875 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.438876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.439030 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.439093 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.439329 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.439412 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 19:20:52.439600 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.430856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35685 (local address 127.9.91.65:48099)
0809 19:20:52.431050 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.431087 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.432122 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.432329 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.432332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.432588 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:52.432770 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.432775 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.433467 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.433468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.434304 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.434306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.434653 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.434654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.434757 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.435236 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:52.435244 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:52.436584 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:52.438226 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:52.438234 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:52.438236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:52.438523 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:52.438723 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:52.438723 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:52.438723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:52.438822 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:52.439100 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:52.439105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:52.439220 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.439301 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.439422 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260809 19:20:52.440117 10722 heartbeater.cc:347] Connected to a master server at 127.9.91.126:35685
I20260809 19:20:52.440217 10722 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:52.440397 10722 heartbeater.cc:511] Master 127.9.91.126:35685 requested a full tablet report, sending...
I20260809 19:20:52.440984 10428 ts_manager.cc:194] Registered new tserver with Master: 5524f691bc234bb88eb8e70acdec2842 (127.9.91.65:44631)
I20260809 19:20:52.441825 10428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:48099
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:20:52.461635 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35685
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:52.541218 10733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:52.541390 10733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:52.541411 10733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:52.542464 10733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:52.542495 10733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:52.542520 10733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:20:52.543947 10733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:52.544127 10733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:52.544309 10733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:52.548780 10743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:52.548826 10740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:52.548943 10733 server_base.cc:1034] running on GCE node
W20260809 19:20:52.548794 10739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:52.549193 10733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:52.549415 10733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:52.550554 10733 hybrid_clock.cc:648] HybridClock initialized: now 1786303252550540 us; error 26 us; skew 500 ppm
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:52.552851 10733 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:20:52.553296 10733 webserver.cc:460] Webserver started at http://127.9.91.66:44109/ using document root <none> and password file <none>
I20260809 19:20:52.553499 10733 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:52.553541 10733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:52.553666 10733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:52.554610 10733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "8becdaec8159c7111e5da35622d53114"
server_key_iv: "712358969c07b8afa26f91f5699d0897"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.554939 10733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "8becdaec8159c7111e5da35622d53114"
server_key_iv: "712358969c07b8afa26f91f5699d0897"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.556185 10733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:52.556828 10755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.556962 10733 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:52.557035 10733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "8becdaec8159c7111e5da35622d53114"
server_key_iv: "712358969c07b8afa26f91f5699d0897"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.557098 10733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:52.571352 10733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:52.572932 10733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:52.573051 10733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:52.573252 10733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:52.573526 10733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:52.573559 10733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.573580 10733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:52.573602 10733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.579312 10733 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:45289
I20260809 19:20:52.579373 10918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:45289 every 8 connection(s)
I20260809 19:20:52.579636 10733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:52.586076 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10733
I20260809 19:20:52.586182 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:20:52.586349 9581 external_mini_cluster.cc:1428] Setting key a1c6f0c6ab73ed3b3477897c08ff1b3e
I20260809 19:20:52.588145 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.580376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:35557 (local address 127.9.91.126:35685)
0809 19:20:52.580457 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.580460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.580620 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:20:52.581466 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.581466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.581477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.581530 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.582082 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.582561 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.583550 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.583734 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.583783 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.585607 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.585609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.585609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.585615 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.586985 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.587336 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.587337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.587337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.587399 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.587592 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.587593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.587593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.587715 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.587774 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.587976 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.588064 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:20:52.588201 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.580346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35685 (local address 127.9.91.66:35557)
0809 19:20:52.580512 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.580545 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.581378 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.581602 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.581606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.581841 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:52.582015 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.582019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.582621 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.582622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.583484 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.583487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.583791 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.583792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.583888 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.584273 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:52.584280 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:52.585526 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:52.587061 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:52.587067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:52.587069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:52.587271 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:52.587458 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:52.587459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:52.587459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:52.587544 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:52.587771 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:52.587774 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:52.587874 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.587944 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.588051 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260809 19:20:52.588615 10920 heartbeater.cc:347] Connected to a master server at 127.9.91.126:35685
I20260809 19:20:52.588722 10920 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:52.588899 10920 heartbeater.cc:511] Master 127.9.91.126:35685 requested a full tablet report, sending...
I20260809 19:20:52.589258 10428 ts_manager.cc:194] Registered new tserver with Master: 7fc19d362b9f44a086ea5ce9b04fe7a2 (127.9.91.66:45289)
I20260809 19:20:52.589602 10428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:35557
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:20:52.609305 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35685
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:52.688161 10930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:52.688345 10930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:52.688367 10930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:52.689436 10930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:52.689468 10930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:52.689492 10930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:20:52.690902 10930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:52.691123 10930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:52.691287 10930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:52.695832 10940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:52.695832 10938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:52.695854 10936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:52.696120 10930 server_base.cc:1034] running on GCE node
I20260809 19:20:52.696354 10930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:52.696588 10930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:52.697721 10930 hybrid_clock.cc:648] HybridClock initialized: now 1786303252697706 us; error 27 us; skew 500 ppm
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:52.700112 10930 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:20:52.700491 10930 webserver.cc:460] Webserver started at http://127.9.91.67:45317/ using document root <none> and password file <none>
I20260809 19:20:52.700691 10930 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:52.700736 10930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:52.700858 10930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:52.701782 10930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "3819f4ff03a64166a53a6977d47d91ef"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "cc411b1229934c8ed0529655a17dee96"
server_key_iv: "2c75744ca353834e815917c3a08f4ce0"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.702112 10930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "3819f4ff03a64166a53a6977d47d91ef"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "cc411b1229934c8ed0529655a17dee96"
server_key_iv: "2c75744ca353834e815917c3a08f4ce0"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.703356 10930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:20:52.703966 10948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.704146 10930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:20:52.704227 10930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "3819f4ff03a64166a53a6977d47d91ef"
format_stamp: "Formatted at 2026-08-09 19:20:52 on dist-test-slave-1vmg"
server_key: "cc411b1229934c8ed0529655a17dee96"
server_key_iv: "2c75744ca353834e815917c3a08f4ce0"
server_key_version: "encryptionkey@0"
I20260809 19:20:52.704285 10930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:52.716579 10930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:52.718384 10930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:52.718509 10930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:52.718711 10930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:20:52.719038 10930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:52.719074 10930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.719099 10930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:52.719123 10930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:52.725756 10930 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:33007
I20260809 19:20:52.725793 11113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:33007 every 8 connection(s)
I20260809 19:20:52.726073 10930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:52.733541 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10930
I20260809 19:20:52.733639 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:20:52.733820 9581 external_mini_cluster.cc:1428] Setting key e66b313803b966a4fa78bc7f8b57c4bc
I20260809 19:20:52.734598 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.726700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:36993 (local address 127.9.91.126:35685)
0809 19:20:52.726771 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.726773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.726944 (+ 171us) server_negotiation.cc:407] Connection header received
0809 19:20:52.727838 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.727839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.727852 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.727912 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.728496 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.729004 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.729970 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.730153 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.730201 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.732159 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.732160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.732161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.732168 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.733209 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.733573 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.733574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.733574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.733635 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.733943 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.733944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.733945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.734060 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.734119 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.734407 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.734495 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:20:52.734727 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.726675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35685 (local address 127.9.91.67:36993)
0809 19:20:52.726826 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.726859 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.727725 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.727973 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.727976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.728223 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:52.728398 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.728401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.729106 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.729107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.729901 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.729904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.730209 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.730210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.730309 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.730693 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:52.730700 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:52.732072 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:52.733275 (+ 1203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:52.733282 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:52.733283 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:52.733490 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:52.733728 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:52.733729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:52.733729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:52.733876 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:52.734112 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:52.734118 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:52.734268 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.734374 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.734522 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260809 19:20:52.735162 11115 heartbeater.cc:347] Connected to a master server at 127.9.91.126:35685
I20260809 19:20:52.735248 11115 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:52.735414 11115 heartbeater.cc:511] Master 127.9.91.126:35685 requested a full tablet report, sending...
I20260809 19:20:52.735733 10428 ts_manager.cc:194] Registered new tserver with Master: 3819f4ff03a64166a53a6977d47d91ef (127.9.91.67:33007)
I20260809 19:20:52.736073 10428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:36993
I20260809 19:20:52.745370 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:20:52.752107 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.746826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.9.91.126:35685)
0809 19:20:52.746896 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.746898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.747066 (+ 168us) server_negotiation.cc:407] Connection header received
0809 19:20:52.747126 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.747127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.747137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.747185 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.747659 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.748134 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.748924 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.749110 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.749173 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.750112 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.750113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.750114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.750120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.751221 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.751544 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.751545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.751545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.751589 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.751721 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.751721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.751721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.751842 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.751912 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.751989 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.752046 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 19:20:52.754084 10428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:20:52.755698 10428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:52.762225 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.757980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:45289 (local address 127.0.0.1:47892)
0809 19:20:52.758118 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.758157 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.758221 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.759093 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.759093 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.759096 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:52.759147 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.759148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.760053 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.760054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.761225 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.761227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.762013 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.762014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.762071 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.762080 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.762139 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.762175 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":116,"thread_start_us":94,"threads_started":1}
I20260809 19:20:52.762269 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.758150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47892 (local address 127.9.91.66:45289)
0809 19:20:52.758821 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.758826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.758835 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:20:52.758872 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.758874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.758888 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.759015 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:52.759214 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.759957 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.761302 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.761940 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.762024 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.762081 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.762152 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.762192 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":79,"threads_started":1}
I20260809 19:20:52.762526 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.757981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:44631 (local address 127.0.0.1:60244)
0809 19:20:52.758118 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.758157 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.758221 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.758697 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.758700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.758702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:52.758761 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.758762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.760100 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.760101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.761446 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.761450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.762305 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.762306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.762372 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.762376 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.762427 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.762464 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":68,"thread_start_us":41,"threads_started":1}
I20260809 19:20:52.762558 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.758359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.9.91.65:44631)
0809 19:20:52.758543 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.758547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.758554 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:20:52.758585 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.758586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.758598 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.758676 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:52.758835 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.759966 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.761538 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.762222 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.762289 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.762345 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.762442 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.762483 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260809 19:20:52.763315 10823 tablet_service.cc:1467] Processing CreateTablet for tablet d7ea5c8ae9884da39af7cef4e14523ad (DEFAULT_TABLE table=test-table [id=1dc812cdfb8644a0b70de8886eba8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:52.763447 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.758175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:33007 (local address 127.0.0.1:33772)
0809 19:20:52.758369 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.758379 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.758432 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.758887 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.758888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.758890 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:52.758957 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.758959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.760283 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.760283 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.761460 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.761462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.763215 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.763216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.763280 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.763284 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.763341 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.763381 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":5,"thread_start_us":64,"threads_started":1}
I20260809 19:20:52.763341 10626 tablet_service.cc:1467] Processing CreateTablet for tablet d7ea5c8ae9884da39af7cef4e14523ad (DEFAULT_TABLE table=test-table [id=1dc812cdfb8644a0b70de8886eba8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:52.763499 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.758354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33772 (local address 127.9.91.67:33007)
0809 19:20:52.758513 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.758518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.758528 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:20:52.758558 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.758559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.758571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.758633 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:52.759111 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.760186 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.762380 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.763063 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.763131 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.763179 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.763372 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.763417 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 19:20:52.763687 10626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ea5c8ae9884da39af7cef4e14523ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:52.763692 10823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ea5c8ae9884da39af7cef4e14523ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:52.764053 11014 tablet_service.cc:1467] Processing CreateTablet for tablet d7ea5c8ae9884da39af7cef4e14523ad (DEFAULT_TABLE table=test-table [id=1dc812cdfb8644a0b70de8886eba8ecd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:52.764333 11014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ea5c8ae9884da39af7cef4e14523ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:52.766239 11140 tablet_bootstrap.cc:492] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842: Bootstrap starting.
I20260809 19:20:52.766748 11140 tablet_bootstrap.cc:654] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:52.766955 11141 tablet_bootstrap.cc:492] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2: Bootstrap starting.
I20260809 19:20:52.767338 11140 log.cc:824] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:52.767604 11141 tablet_bootstrap.cc:654] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:52.767792 11142 tablet_bootstrap.cc:492] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef: Bootstrap starting.
I20260809 19:20:52.768079 11140 tablet_bootstrap.cc:492] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842: No bootstrap required, opened a new log
I20260809 19:20:52.768157 11140 ts_tablet_manager.cc:1397] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:20:52.768162 11141 log.cc:824] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:52.768607 11142 tablet_bootstrap.cc:654] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:52.769021 11141 tablet_bootstrap.cc:492] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2: No bootstrap required, opened a new log
I20260809 19:20:52.769124 11141 ts_tablet_manager.cc:1397] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:20:52.769192 11142 log.cc:824] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:52.769855 11142 tablet_bootstrap.cc:492] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef: No bootstrap required, opened a new log
I20260809 19:20:52.769913 11142 ts_tablet_manager.cc:1397] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:20:52.770110 11140 raft_consensus.cc:348] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.770308 11140 raft_consensus.cc:374] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:52.770344 11140 raft_consensus.cc:729] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5524f691bc234bb88eb8e70acdec2842, State: Initialized, Role: FOLLOWER
I20260809 19:20:52.770457 11140 consensus_queue.cc:260] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.770742 11140 ts_tablet_manager.cc:1428] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:20:52.770872 10722 heartbeater.cc:503] Master 127.9.91.126:35685 was elected leader, sending a full tablet report...
I20260809 19:20:52.771394 11141 raft_consensus.cc:348] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.771517 11141 raft_consensus.cc:374] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:52.771536 11141 raft_consensus.cc:729] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fc19d362b9f44a086ea5ce9b04fe7a2, State: Initialized, Role: FOLLOWER
I20260809 19:20:52.771615 11141 consensus_queue.cc:260] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.771879 10920 heartbeater.cc:503] Master 127.9.91.126:35685 was elected leader, sending a full tablet report...
I20260809 19:20:52.772022 11141 ts_tablet_manager.cc:1428] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:20:52.771718 11142 raft_consensus.cc:348] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.772230 11142 raft_consensus.cc:374] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:52.772280 11142 raft_consensus.cc:729] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3819f4ff03a64166a53a6977d47d91ef, State: Initialized, Role: FOLLOWER
I20260809 19:20:52.772385 11142 consensus_queue.cc:260] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.772653 11142 ts_tablet_manager.cc:1428] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:20:52.772660 11115 heartbeater.cc:503] Master 127.9.91.126:35685 was elected leader, sending a full tablet report...
W20260809 19:20:52.830137 10924 tablet.cc:2369] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:52.861852 11154 raft_consensus.cc:482] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:52.861963 11154 raft_consensus.cc:504] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.862345 11154 leader_election.cc:288] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5524f691bc234bb88eb8e70acdec2842 (127.9.91.65:44631), 7fc19d362b9f44a086ea5ce9b04fe7a2 (127.9.91.66:45289)
I20260809 19:20:52.864946 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.862497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:44631 (local address 127.9.91.67:53801)
0809 19:20:52.862539 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.862550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.862589 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.862745 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.862746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.862749 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:52.862817 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.862819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.863471 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.863472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.864194 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.864196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.864719 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.864720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.864790 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.864793 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.864838 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.864875 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 19:20:52.864950 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.862570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:53801 (local address 127.9.91.65:44631)
0809 19:20:52.862622 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.862624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.862630 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:20:52.862652 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.862653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.862665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.862718 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:52.862888 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.863393 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.864284 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.864640 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.864701 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.864713 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.864854 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.864893 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:20:52.865298 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.862559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:44787 (local address 127.9.91.66:45289)
0809 19:20:52.862626 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.862628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.862941 (+ 313us) server_negotiation.cc:407] Connection header received
0809 19:20:52.862991 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.862992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.863006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.863062 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:52.863197 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.863647 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.864550 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.864928 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.864990 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.865000 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.865199 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.865238 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:20:52.865314 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.862497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:45289 (local address 127.9.91.67:44787)
0809 19:20:52.862804 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:52.862812 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:52.862936 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:52.863065 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:52.863066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:52.863067 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:52.863137 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.863138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.863722 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.863723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.864462 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:52.864464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.865023 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.865024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:52.865096 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.865102 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 19:20:52.865185 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:20:52.865223 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":138,"thread_start_us":107,"threads_started":1}
I20260809 19:20:52.865281 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ea5c8ae9884da39af7cef4e14523ad" candidate_uuid: "3819f4ff03a64166a53a6977d47d91ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5524f691bc234bb88eb8e70acdec2842" is_pre_election: true
I20260809 19:20:52.865415 10656 raft_consensus.cc:2454] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3819f4ff03a64166a53a6977d47d91ef in term 0.
I20260809 19:20:52.865669 10949 leader_election.cc:302] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3819f4ff03a64166a53a6977d47d91ef, 5524f691bc234bb88eb8e70acdec2842; no voters:
I20260809 19:20:52.865656 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ea5c8ae9884da39af7cef4e14523ad" candidate_uuid: "3819f4ff03a64166a53a6977d47d91ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" is_pre_election: true
I20260809 19:20:52.865782 10848 raft_consensus.cc:2454] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3819f4ff03a64166a53a6977d47d91ef in term 0.
I20260809 19:20:52.865795 11154 raft_consensus.cc:2781] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:52.865847 11154 raft_consensus.cc:482] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:52.865878 11154 raft_consensus.cc:3037] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:52.866657 11154 raft_consensus.cc:504] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.866783 11154 leader_election.cc:288] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [CANDIDATE]: Term 1 election: Requested vote from peers 5524f691bc234bb88eb8e70acdec2842 (127.9.91.65:44631), 7fc19d362b9f44a086ea5ce9b04fe7a2 (127.9.91.66:45289)
I20260809 19:20:52.866955 10656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ea5c8ae9884da39af7cef4e14523ad" candidate_uuid: "3819f4ff03a64166a53a6977d47d91ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5524f691bc234bb88eb8e70acdec2842"
I20260809 19:20:52.867040 10656 raft_consensus.cc:3037] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:52.867818 10656 raft_consensus.cc:2454] T d7ea5c8ae9884da39af7cef4e14523ad P 5524f691bc234bb88eb8e70acdec2842 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3819f4ff03a64166a53a6977d47d91ef in term 1.
I20260809 19:20:52.867913 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ea5c8ae9884da39af7cef4e14523ad" candidate_uuid: "3819f4ff03a64166a53a6977d47d91ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2"
I20260809 19:20:52.867966 10949 leader_election.cc:302] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3819f4ff03a64166a53a6977d47d91ef, 5524f691bc234bb88eb8e70acdec2842; no voters:
I20260809 19:20:52.867991 10848 raft_consensus.cc:3037] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:52.868084 11154 raft_consensus.cc:2781] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:52.868263 11154 raft_consensus.cc:686] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [term 1 LEADER]: Becoming Leader. State: Replica: 3819f4ff03a64166a53a6977d47d91ef, State: Running, Role: LEADER
I20260809 19:20:52.868359 11154 consensus_queue.cc:237] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } }
I20260809 19:20:52.868908 10848 raft_consensus.cc:2454] T d7ea5c8ae9884da39af7cef4e14523ad P 7fc19d362b9f44a086ea5ce9b04fe7a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3819f4ff03a64166a53a6977d47d91ef in term 1.
I20260809 19:20:52.869385 10428 catalog_manager.cc:5495] T d7ea5c8ae9884da39af7cef4e14523ad P 3819f4ff03a64166a53a6977d47d91ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 3819f4ff03a64166a53a6977d47d91ef (127.9.91.67). New cstate: current_term: 1 leader_uuid: "3819f4ff03a64166a53a6977d47d91ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5524f691bc234bb88eb8e70acdec2842" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fc19d362b9f44a086ea5ce9b04fe7a2" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 45289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3819f4ff03a64166a53a6977d47d91ef" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 33007 } health_report { overall_health: HEALTHY } } }
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:20:52.881170 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.874591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.9.91.65:44631)
0809 19:20:52.874630 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.874631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.874785 (+ 154us) server_negotiation.cc:407] Connection header received
0809 19:20:52.874831 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.874832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.874842 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.874885 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.875516 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.875942 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.876609 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.876790 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.876842 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.879044 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.879052 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.879053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.879059 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.880304 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.880603 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.880605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.880606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.880657 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.880786 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.880787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.880787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.880902 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.880994 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.881046 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.881104 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:20:52.902889 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.896997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.9.91.65:44631)
0809 19:20:52.897050 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.897052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.897130 (+ 78us) server_negotiation.cc:407] Connection header received
0809 19:20:52.897181 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.897182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.897191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.897232 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.897688 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.898126 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.898791 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.899033 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.899084 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.901028 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.901029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.901029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.901035 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.902060 (+ 1025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.902322 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.902323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.902323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.902362 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.902533 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.902534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.902534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.902606 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.902670 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.902774 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.902825 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:20:52 dist-test-slave-1vmg krb5kdc[10364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:20:52.923857 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:52.917790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.9.91.65:44631)
0809 19:20:52.917865 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:52.917866 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:52.917978 (+ 112us) server_negotiation.cc:407] Connection header received
0809 19:20:52.918052 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:52.918053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:52.918062 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:52.918102 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:52.918579 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.919064 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.919735 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:52.919931 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:52.919984 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:52.921983 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:52.921984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:52.921984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:52.921990 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:52.923115 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.923388 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.923389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.923389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.923431 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:52.923551 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:52.923552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:52.923552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:52.923624 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:52.923683 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:52.923752 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:20:52.923801 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 19:20:52.924252 10599 tablet_service.cc:2908] Rejecting scan request for tablet d7ea5c8ae9884da39af7cef4e14523ad: Uninitialized: clean time has not yet been initialized
I20260809 19:20:52.924947 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10533
I20260809 19:20:52.941900 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10733
I20260809 19:20:52.948777 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10930
I20260809 19:20:52.959245 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10382
2026-08-09T19:20:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2109 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:20:53 dist-test-slave-1vmg krb5kdc[11177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:20:53 dist-test-slave-1vmg krb5kdc[11177](info): set up 2 sockets
Aug 09 19:20:53 dist-test-slave-1vmg krb5kdc[11177](info): commencing operation
krb5kdc: starting...
W20260809 19:20:54.088650 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:20:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:20:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:20:54.196664 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:34731
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:34731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:54.277091 11195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:54.277282 11195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:54.277305 11195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:54.278434 11195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:20:54.278478 11195 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:20:54.278492 11195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:54.278507 11195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:54.278534 11195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:20:54.278563 11195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:20:54.280081 11195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:34731
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:34731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:54.280299 11195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:54.280514 11195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:54.285108 11205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.285120 11204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.285120 11207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:54.285305 11195 server_base.cc:1034] running on GCE node
I20260809 19:20:54.285646 11195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:54.285928 11195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:54.287072 11195 hybrid_clock.cc:648] HybridClock initialized: now 1786303254287062 us; error 27 us; skew 500 ppm
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:54.294054 11195 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:20:54.294579 11195 webserver.cc:460] Webserver started at http://127.9.91.126:34561/ using document root <none> and password file <none>
I20260809 19:20:54.294782 11195 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:54.294833 11195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:54.295001 11195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:54.296021 11195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "10b0538b497d4e9e82d69db7db8b947c"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "40d937f214056f9c0999cb4b7b1c4ec1"
server_key_iv: "93b153d3cb5f305ee8f5f0cf4501d671"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.296368 11195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "10b0538b497d4e9e82d69db7db8b947c"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "40d937f214056f9c0999cb4b7b1c4ec1"
server_key_iv: "93b153d3cb5f305ee8f5f0cf4501d671"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.297668 11195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:20:54.298328 11219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.298482 11195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:20:54.298579 11195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "10b0538b497d4e9e82d69db7db8b947c"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "40d937f214056f9c0999cb4b7b1c4ec1"
server_key_iv: "93b153d3cb5f305ee8f5f0cf4501d671"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.298645 11195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:54.317054 11195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:54.318650 11195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:54.318787 11195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:54.323925 11195 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:34731
I20260809 19:20:54.323963 11314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:34731 every 8 connection(s)
I20260809 19:20:54.324261 11195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:20:54.324882 11315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:54.327073 11315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c: Bootstrap starting.
I20260809 19:20:54.327618 11315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:54.327893 11315 log.cc:824] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:54.328516 11315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c: No bootstrap required, opened a new log
I20260809 19:20:54.330681 11315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b0538b497d4e9e82d69db7db8b947c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34731 } }
I20260809 19:20:54.330818 11315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:54.330839 11315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10b0538b497d4e9e82d69db7db8b947c, State: Initialized, Role: FOLLOWER
I20260809 19:20:54.330981 11315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b0538b497d4e9e82d69db7db8b947c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34731 } }
I20260809 19:20:54.331050 11315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:20:54.331072 11315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:20:54.331117 11315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:54.331113 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11195
I20260809 19:20:54.331190 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:20:54.331338 9581 external_mini_cluster.cc:1428] Setting key 6af31dd83e2f45b623b3e161513664eb
I20260809 19:20:54.331677 11315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b0538b497d4e9e82d69db7db8b947c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34731 } }
I20260809 19:20:54.331779 11315 leader_election.cc:302] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10b0538b497d4e9e82d69db7db8b947c; no voters:
I20260809 19:20:54.331936 11315 leader_election.cc:288] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:20:54.331962 11321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:54.332103 11321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [term 1 LEADER]: Becoming Leader. State: Replica: 10b0538b497d4e9e82d69db7db8b947c, State: Running, Role: LEADER
I20260809 19:20:54.332183 11315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:20:54.332190 11321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b0538b497d4e9e82d69db7db8b947c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34731 } }
I20260809 19:20:54.332587 11321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10b0538b497d4e9e82d69db7db8b947c. Latest consensus state: current_term: 1 leader_uuid: "10b0538b497d4e9e82d69db7db8b947c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b0538b497d4e9e82d69db7db8b947c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34731 } } }
I20260809 19:20:54.332679 11321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:54.332584 11326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10b0538b497d4e9e82d69db7db8b947c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10b0538b497d4e9e82d69db7db8b947c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34731 } } }
I20260809 19:20:54.332844 11326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:54.332882 11334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:20:54.333398 11334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:20:54.335768 11334 catalog_manager.cc:1347] Generated new cluster ID: 23b7f68a6e964f63b00519879871d8ac
I20260809 19:20:54.335819 11334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:54.342345 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.331767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.9.91.126:34731)
0809 19:20:54.331940 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.331949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.331981 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:20:54.332107 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.332114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.332173 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.333105 (+ 932us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.333219 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.334413 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.335536 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.336164 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.336269 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.338843 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.338847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.338847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.338858 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.340813 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.341206 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.341208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.341209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.341312 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.341501 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.341501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.341502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.341765 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.341903 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.341998 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.342133 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260809 19:20:54.343663 11334 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:20:54.344360 11334 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:20:54.349562 11334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c: Generated new TSK 0
I20260809 19:20:54.349720 11334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:20:54.378793 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34731
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:54.458988 11354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:54.459177 11354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:54.459198 11354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:54.460340 11354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:54.460379 11354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:54.460400 11354 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:20:54.460431 11354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:20:54.461861 11354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:34731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:54.462085 11354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:54.462282 11354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:54.466878 11362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.466895 11363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:54.466894 11354 server_base.cc:1034] running on GCE node
W20260809 19:20:54.466861 11365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:54.467234 11354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:54.467461 11354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:54.468681 11354 hybrid_clock.cc:648] HybridClock initialized: now 1786303254468665 us; error 30 us; skew 500 ppm
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:54.471056 11354 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:20:54.471455 11354 webserver.cc:460] Webserver started at http://127.9.91.65:42883/ using document root <none> and password file <none>
I20260809 19:20:54.471664 11354 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:54.471710 11354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:54.471832 11354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:54.472792 11354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "f80e11237ba54b81833627d0bf2dcb48"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "d597e5bd0fde403bb15ad9a5b471e1a1"
server_key_iv: "ab3d6bd4713293450f665a636b633663"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.473121 11354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "f80e11237ba54b81833627d0bf2dcb48"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "d597e5bd0fde403bb15ad9a5b471e1a1"
server_key_iv: "ab3d6bd4713293450f665a636b633663"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.474398 11354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:20:54.475040 11376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.475194 11354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.475277 11354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "f80e11237ba54b81833627d0bf2dcb48"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "d597e5bd0fde403bb15ad9a5b471e1a1"
server_key_iv: "ab3d6bd4713293450f665a636b633663"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.475343 11354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:54.486557 11354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:54.488163 11354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:54.488296 11354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:54.488736 11354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:54.488773 11354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.488798 11354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:54.488816 11354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:54.496013 11354 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:38697
I20260809 19:20:54.496248 11538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:38697 every 8 connection(s)
I20260809 19:20:54.496413 11354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:20:54.498230 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.489269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35407 (local address 127.9.91.126:34731)
0809 19:20:54.489335 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.489337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.489532 (+ 195us) server_negotiation.cc:407] Connection header received
0809 19:20:54.490136 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.490138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.490152 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.490189 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.491004 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.491537 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.492945 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.493177 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.493233 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.495722 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.495723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.495723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.495731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.497184 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.497540 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.497541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.497541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.497596 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.497780 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.497780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.497780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.497873 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.497915 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.498088 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.498155 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:20:54.498349 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.489236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34731 (local address 127.9.91.65:35407)
0809 19:20:54.489401 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.489447 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.490037 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.490270 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.490274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.490609 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:54.490897 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.490903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.491608 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.491609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.492857 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.492861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.493252 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.493253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.493394 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.493953 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:54.493962 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:54.495600 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:54.497250 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.497253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.497255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.497476 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.497647 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.497647 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.497656 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.497744 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.497907 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:54.497909 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:54.498007 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.498075 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.498184 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260809 19:20:54.498842 11539 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34731
I20260809 19:20:54.498965 11539 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:54.499162 11539 heartbeater.cc:511] Master 127.9.91.126:34731 requested a full tablet report, sending...
I20260809 19:20:54.499717 11246 ts_manager.cc:194] Registered new tserver with Master: f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697)
I20260809 19:20:54.500517 11246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:35407
I20260809 19:20:54.503032 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11354
I20260809 19:20:54.503132 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:20:54.503306 9581 external_mini_cluster.cc:1428] Setting key ffbdcf9725f46a119b70f38f9e5bcb8b
I20260809 19:20:54.504865 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.499909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:60179 (local address 127.9.91.126:34731)
0809 19:20:54.499970 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.499971 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.500034 (+ 63us) server_negotiation.cc:407] Connection header received
0809 19:20:54.500076 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.500077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.500087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.500146 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.500400 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.501033 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.501732 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.501906 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.501958 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.502658 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.502659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.502660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.502665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.503920 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.504214 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.504215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.504215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.504266 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.504427 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.504428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.504428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.504524 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.504576 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.504716 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.504785 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:20:54.504873 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.499860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34731 (local address 127.9.91.65:60179)
0809 19:20:54.499984 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.499993 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.500042 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.500151 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.500151 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.500300 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:54.500350 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.500351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.501117 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.501117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.501666 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.501668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.501965 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.501966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.502019 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.502330 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:54.502335 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:54.502585 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:54.503993 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.503994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.503994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.504154 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.504317 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.504318 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.504318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.504377 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.504576 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:54.504577 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:54.504622 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.504686 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.504781 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:20:54.526865 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34731
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:54.606199 11548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:54.606374 11548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:54.606395 11548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:54.607533 11548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:54.607573 11548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:54.607590 11548 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:20:54.607606 11548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:20:54.608973 11548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:34731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:54.609189 11548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:54.609390 11548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:54.614007 11561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.614032 11560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.614032 11564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:54.614199 11548 server_base.cc:1034] running on GCE node
I20260809 19:20:54.614379 11548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:54.614614 11548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:54.615746 11548 hybrid_clock.cc:648] HybridClock initialized: now 1786303254615732 us; error 27 us; skew 500 ppm
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:54.618178 11548 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:20:54.618584 11548 webserver.cc:460] Webserver started at http://127.9.91.66:45559/ using document root <none> and password file <none>
I20260809 19:20:54.618793 11548 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:54.618839 11548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:54.618994 11548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:54.620008 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "73f142ab1f0245c78e33b0513bc6b047"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "fe86aa21047bcca7d23687e19ee45051"
server_key_iv: "5b04fb50153d9c3ef1293ddda2173bae"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.620333 11548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "73f142ab1f0245c78e33b0513bc6b047"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "fe86aa21047bcca7d23687e19ee45051"
server_key_iv: "5b04fb50153d9c3ef1293ddda2173bae"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.621630 11548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:20:54.622262 11574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.622407 11548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:54.622486 11548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "73f142ab1f0245c78e33b0513bc6b047"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "fe86aa21047bcca7d23687e19ee45051"
server_key_iv: "5b04fb50153d9c3ef1293ddda2173bae"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.622545 11548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:54.631417 11548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:54.633335 11548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:54.633456 11548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:54.633919 11548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:54.633958 11548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.633983 11548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:54.634001 11548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:54.641322 11548 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:36813
I20260809 19:20:54.641396 11725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:36813 every 8 connection(s)
I20260809 19:20:54.641665 11548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:20:54.641985 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.634344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:39727 (local address 127.9.91.126:34731)
0809 19:20:54.634411 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.634414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.634598 (+ 184us) server_negotiation.cc:407] Connection header received
0809 19:20:54.635021 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.635022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.635037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.635113 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.635657 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.636142 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.637127 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.637371 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.637427 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.639418 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.639420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.639420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.639427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.640728 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.641098 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.641099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.641099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.641161 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.641384 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.641385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.641385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.641497 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.641551 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.641822 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.641898 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:20:54.642135 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.634280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34731 (local address 127.9.91.66:39727)
0809 19:20:54.634497 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.634522 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.634906 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.635165 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.635168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.635377 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:54.635576 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.635580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.636229 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.636230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.637047 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.637050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.637441 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.637442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.637584 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.638021 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:54.638028 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:54.639314 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:54.640798 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.640804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.640806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.641032 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.641210 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.641210 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.641211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.641330 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.641552 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:54.641559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:54.641698 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.641798 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.641940 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260809 19:20:54.642614 11726 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34731
I20260809 19:20:54.642693 11726 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:54.642850 11726 heartbeater.cc:511] Master 127.9.91.126:34731 requested a full tablet report, sending...
I20260809 19:20:54.643211 11247 ts_manager.cc:194] Registered new tserver with Master: 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813)
I20260809 19:20:54.643570 11247 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:39727
I20260809 19:20:54.648272 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.643856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:54701 (local address 127.9.91.126:34731)
0809 19:20:54.643899 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.643901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.643993 (+ 92us) server_negotiation.cc:407] Connection header received
0809 19:20:54.644019 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.644020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.644030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.644090 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.644359 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.644791 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.645395 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.645556 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.645594 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.646271 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.646272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.646272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.646278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.647452 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.647732 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.647732 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.647733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.647775 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.647897 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.647897 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.647898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.648016 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.648062 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.648166 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.648217 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:20:54.648280 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.643793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34731 (local address 127.9.91.66:54701)
0809 19:20:54.643929 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.643939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.643988 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.644100 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.644100 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.644238 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:54.644294 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.644295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.644856 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.644857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.645343 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.645344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.645601 (+ 257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.645602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.645649 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.645956 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:54.645960 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:54.646208 (+ 248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:54.647528 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.647528 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.647529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.647677 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.647816 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.647816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.647817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.647860 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.648057 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:54.648058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:54.648106 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.648150 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.648220 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260809 19:20:54.651227 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11548
I20260809 19:20:54.651307 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:20:54.651437 9581 external_mini_cluster.cc:1428] Setting key d4ac800b2e51e68df81cadcbb4ce7a7b
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:20:54.674687 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34731
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:54.753093 11743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:54.753293 11743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:54.753326 11743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:54.754463 11743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:54.754505 11743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:54.754532 11743 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:20:54.754554 11743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:20:54.756035 11743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:34731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:54.756289 11743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:54.756536 11743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:54.761282 11752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.761304 11755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:54.761273 11753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:54.761523 11743 server_base.cc:1034] running on GCE node
I20260809 19:20:54.761714 11743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:54.761927 11743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:54.763063 11743 hybrid_clock.cc:648] HybridClock initialized: now 1786303254763050 us; error 28 us; skew 500 ppm
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:54.765413 11743 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:20:54.765798 11743 webserver.cc:460] Webserver started at http://127.9.91.67:45575/ using document root <none> and password file <none>
I20260809 19:20:54.766002 11743 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:54.766049 11743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:54.766178 11743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:54.767181 11743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "cc82b16658ea4335b82ddbf9d5567ba8"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "9e5cec8f6443b2c9bbe87d7e73d90762"
server_key_iv: "dd4ca0490fe4a4e6e51e55a54e56eca3"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.767495 11743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "cc82b16658ea4335b82ddbf9d5567ba8"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "9e5cec8f6443b2c9bbe87d7e73d90762"
server_key_iv: "dd4ca0490fe4a4e6e51e55a54e56eca3"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.768747 11743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:20:54.769364 11765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.769505 11743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:54.769584 11743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "cc82b16658ea4335b82ddbf9d5567ba8"
format_stamp: "Formatted at 2026-08-09 19:20:54 on dist-test-slave-1vmg"
server_key: "9e5cec8f6443b2c9bbe87d7e73d90762"
server_key_iv: "dd4ca0490fe4a4e6e51e55a54e56eca3"
server_key_version: "encryptionkey@0"
I20260809 19:20:54.769640 11743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:54.784301 11743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:54.786290 11743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:54.786425 11743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:54.786897 11743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:54.786964 11743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:54.787003 11743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:54.787027 11743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:54.794143 11743 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:41523
I20260809 19:20:54.794274 11926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:41523 every 8 connection(s)
I20260809 19:20:54.794510 11743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:20:54.796819 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.787431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:41199 (local address 127.9.91.126:34731)
0809 19:20:54.787479 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.787481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.787724 (+ 243us) server_negotiation.cc:407] Connection header received
0809 19:20:54.788229 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.788231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.788245 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.788298 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.789160 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.789917 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.791347 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.791652 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.791739 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.794313 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.794315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.794315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.794324 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.795693 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.796073 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.796074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.796074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.796138 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.796327 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.796327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.796327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.796428 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.796475 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.796683 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.796751 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:20:54.796934 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.787424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34731 (local address 127.9.91.67:41199)
0809 19:20:54.787602 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.787636 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.788120 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.788373 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.788377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.788706 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:54.788981 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.788986 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.790036 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.790038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.791249 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.791253 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.791762 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.791763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.791906 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.792450 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:54.792458 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:54.794190 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:54.795773 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.795779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.795781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.795988 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.796195 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.796196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.796196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.796283 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.796465 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:54.796467 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:54.796580 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.796655 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.796769 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260809 19:20:54.797402 11927 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34731
I20260809 19:20:54.797489 11927 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:54.797686 11927 heartbeater.cc:511] Master 127.9.91.126:34731 requested a full tablet report, sending...
I20260809 19:20:54.798049 11246 ts_manager.cc:194] Registered new tserver with Master: cc82b16658ea4335b82ddbf9d5567ba8 (127.9.91.67:41523)
I20260809 19:20:54.798508 11246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:41199
I20260809 19:20:54.798797 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11743
I20260809 19:20:54.798928 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:20:54.799650 9581 external_mini_cluster.cc:1428] Setting key b476c6a54e6998e391c2575459f32d48
I20260809 19:20:54.800443 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:20:54.804225 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.798439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34731 (local address 127.9.91.67:42479)
0809 19:20:54.798616 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.798626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.798702 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.798820 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.798821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.799051 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:54.799114 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.799116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.799798 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.799800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.800584 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.800594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.800966 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.800967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.801059 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.801422 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:54.801427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:54.801726 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:54.803231 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.803232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.803233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.803388 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.803554 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:54.803554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:54.803555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:54.803626 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:54.803910 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:54.803910 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:54.803974 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.804041 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.804132 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260809 19:20:54.804483 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.798507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:42479 (local address 127.9.91.126:34731)
0809 19:20:54.798551 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.798553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.798677 (+ 124us) server_negotiation.cc:407] Connection header received
0809 19:20:54.798754 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.798755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.798766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.798813 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.799187 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.799679 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.800660 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.800898 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.800947 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.801799 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.801801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.801801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.801808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.803151 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.803445 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.803446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.803447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.803498 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.803697 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.803698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.803699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.803833 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.803904 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.804107 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.804218 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:20:54 dist-test-slave-1vmg krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:54.823488 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.817473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53914 (local address 127.9.91.126:34731)
0809 19:20:54.817525 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.817527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.817535 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:20:54.817600 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.817601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.817611 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.817653 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:54.818153 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.818598 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.819307 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.819485 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.819538 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.821433 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:54.821435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:54.821435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:54.821442 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:54.822466 (+ 1024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.822784 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.822785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.822785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.822835 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:54.823042 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:54.823042 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:54.823043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:54.823145 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:54.823229 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.823339 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.823406 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:20:54.824836 11246 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:20:54.826447 11246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:54.832019 11954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.828463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:41523 (local address 127.0.0.1:55080)
0809 19:20:54.828625 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.828664 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.828729 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.828970 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.828973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.828976 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:54.829102 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.829105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.830060 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.830061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.831009 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.831012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.831738 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.831739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.831822 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.831836 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.831897 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.831951 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":59,"threads_started":1}
I20260809 19:20:54.832206 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.828552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55080 (local address 127.9.91.67:41523)
0809 19:20:54.828736 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.828741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.828751 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:20:54.828810 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.828813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.828827 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.828897 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:54.829185 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.829963 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.831094 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.831664 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.831728 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.831787 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.831917 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.831989 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":91,"threads_started":1}
I20260809 19:20:54.832943 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.828468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38697 (local address 127.0.0.1:58594)
0809 19:20:54.828759 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.828769 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.828822 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.829392 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.829393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.829396 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:54.829436 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.829437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.830684 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.830685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.831590 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.831592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.832736 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.832737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.832793 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.832797 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.832824 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.832879 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":52,"threads_started":1}
I20260809 19:20:54.832839 11839 tablet_service.cc:1467] Processing CreateTablet for tablet 9fa2815ee191423798ecf98c932837c9 (DEFAULT_TABLE table=test-table [id=d4341865fcdc4ba7ab694b5f0929eb6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:54.833182 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fa2815ee191423798ecf98c932837c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:54.833350 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.828577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.9.91.65:38697)
0809 19:20:54.829255 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.829260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.829268 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:20:54.829303 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.829305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.829320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.829395 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:54.829514 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.830606 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.831676 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.832642 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.833115 (+ 473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.833186 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.833212 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.833253 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":85,"threads_started":1}
I20260809 19:20:54.833963 11960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.828661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:36813 (local address 127.0.0.1:53528)
0809 19:20:54.828885 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:54.828893 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:54.829044 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:54.829346 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:54.829347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:54.829349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:54.829396 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.829398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.831667 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.831668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.832742 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:54.832744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.833735 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.833736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:54.833797 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.833802 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:20:54.833865 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 19:20:54.833902 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":94,"threads_started":1}
I20260809 19:20:54.834013 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:54.828903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53528 (local address 127.9.91.66:36813)
0809 19:20:54.829081 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:54.829085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:54.829094 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:20:54.829189 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:54.829192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:54.829208 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:54.829281 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:54.829468 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.830597 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.832830 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:54.833647 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:54.833722 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:54.833779 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:54.833880 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 19:20:54.833919 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260809 19:20:54.833916 11443 tablet_service.cc:1467] Processing CreateTablet for tablet 9fa2815ee191423798ecf98c932837c9 (DEFAULT_TABLE table=test-table [id=d4341865fcdc4ba7ab694b5f0929eb6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:54.834240 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fa2815ee191423798ecf98c932837c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:54.834625 11644 tablet_service.cc:1467] Processing CreateTablet for tablet 9fa2815ee191423798ecf98c932837c9 (DEFAULT_TABLE table=test-table [id=d4341865fcdc4ba7ab694b5f0929eb6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:54.834959 11644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fa2815ee191423798ecf98c932837c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:54.836277 11963 tablet_bootstrap.cc:492] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Bootstrap starting.
I20260809 19:20:54.837026 11963 tablet_bootstrap.cc:654] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:54.837262 11963 log.cc:824] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:54.837852 11966 tablet_bootstrap.cc:492] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Bootstrap starting.
I20260809 19:20:54.837899 11963 tablet_bootstrap.cc:492] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: No bootstrap required, opened a new log
I20260809 19:20:54.837951 11963 ts_tablet_manager.cc:1397] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:20:54.838297 11967 tablet_bootstrap.cc:492] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Bootstrap starting.
I20260809 19:20:54.838447 11966 tablet_bootstrap.cc:654] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:54.838788 11966 log.cc:824] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:54.839058 11967 tablet_bootstrap.cc:654] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:54.839385 11967 log.cc:824] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:54.839677 11966 tablet_bootstrap.cc:492] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: No bootstrap required, opened a new log
I20260809 19:20:54.839762 11966 ts_tablet_manager.cc:1397] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:20:54.840070 11967 tablet_bootstrap.cc:492] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: No bootstrap required, opened a new log
I20260809 19:20:54.840129 11967 ts_tablet_manager.cc:1397] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:20:54.840138 11963 raft_consensus.cc:348] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:54.840303 11963 raft_consensus.cc:374] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:54.840342 11963 raft_consensus.cc:729] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc82b16658ea4335b82ddbf9d5567ba8, State: Initialized, Role: FOLLOWER
I20260809 19:20:54.840469 11963 consensus_queue.cc:260] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:54.840726 11963 ts_tablet_manager.cc:1428] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:20:54.840758 11927 heartbeater.cc:503] Master 127.9.91.126:34731 was elected leader, sending a full tablet report...
I20260809 19:20:54.841601 11966 raft_consensus.cc:348] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:54.841744 11966 raft_consensus.cc:374] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:54.841778 11966 raft_consensus.cc:729] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80e11237ba54b81833627d0bf2dcb48, State: Initialized, Role: FOLLOWER
I20260809 19:20:54.841892 11966 consensus_queue.cc:260] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:54.841997 11967 raft_consensus.cc:348] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:54.842127 11967 raft_consensus.cc:374] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:54.842164 11967 raft_consensus.cc:729] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f142ab1f0245c78e33b0513bc6b047, State: Initialized, Role: FOLLOWER
I20260809 19:20:54.842177 11539 heartbeater.cc:503] Master 127.9.91.126:34731 was elected leader, sending a full tablet report...
I20260809 19:20:54.842262 11967 consensus_queue.cc:260] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:54.842459 11966 ts_tablet_manager.cc:1428] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:20:54.842562 11726 heartbeater.cc:503] Master 127.9.91.126:34731 was elected leader, sending a full tablet report...
I20260809 19:20:54.842587 11967 ts_tablet_manager.cc:1428] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260809 19:20:54.892292 11727 tablet.cc:2369] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:20:54.997221 11540 tablet.cc:2369] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:20:55.045236 11929 tablet.cc:2369] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:55.138401 11975 raft_consensus.cc:482] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:55.138542 11975 raft_consensus.cc:504] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:55.138967 11975 leader_election.cc:288] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697), 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813)
I20260809 19:20:55.142809 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.139285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:37549 (local address 127.9.91.65:38697)
0809 19:20:55.139337 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.139339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.139435 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:20:55.139488 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.139489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.139500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.139546 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:55.139677 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.140418 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.141883 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.142431 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.142502 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.142514 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.142696 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.142738 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:20:55.142823 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.139181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38697 (local address 127.9.91.67:37549)
0809 19:20:55.139369 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:55.139377 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:55.139433 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:55.139559 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:55.139559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:55.139560 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:55.139622 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.139623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.140491 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.140491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.141794 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.141797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.142508 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.142509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.142622 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.142627 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:20:55.142689 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:20:55.142731 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":101,"thread_start_us":107,"threads_started":1}
I20260809 19:20:55.142880 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.139224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:39351 (local address 127.9.91.66:36813)
0809 19:20:55.139268 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.139270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.139376 (+ 106us) server_negotiation.cc:407] Connection header received
0809 19:20:55.139407 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.139408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.139419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.139471 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:55.139644 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.140419 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.141883 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.142464 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.142525 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.142534 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.142782 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.142821 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:20:55.142831 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.139181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:36813 (local address 127.9.91.67:39351)
0809 19:20:55.139241 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:55.139254 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:55.139338 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:55.139491 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:55.139491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:55.139493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:55.139574 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.139576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.140491 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.140491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.141794 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.141797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.142530 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.142531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.142701 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.142704 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:20:55.142749 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:20:55.142781 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":11}
I20260809 19:20:55.143392 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa2815ee191423798ecf98c932837c9" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e11237ba54b81833627d0bf2dcb48" is_pre_election: true
I20260809 19:20:55.143392 11671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa2815ee191423798ecf98c932837c9" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f142ab1f0245c78e33b0513bc6b047" is_pre_election: true
I20260809 19:20:55.143566 11468 raft_consensus.cc:2454] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 0.
I20260809 19:20:55.143589 11671 raft_consensus.cc:2454] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 0.
I20260809 19:20:55.143836 11769 leader_election.cc:302] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc82b16658ea4335b82ddbf9d5567ba8, f80e11237ba54b81833627d0bf2dcb48; no voters:
I20260809 19:20:55.143958 11975 raft_consensus.cc:2781] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:55.144026 11975 raft_consensus.cc:482] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:55.144044 11975 raft_consensus.cc:3037] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.145195 11975 raft_consensus.cc:504] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:55.145386 11975 leader_election.cc:288] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 election: Requested vote from peers f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697), 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813)
I20260809 19:20:55.145478 11671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa2815ee191423798ecf98c932837c9" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f142ab1f0245c78e33b0513bc6b047"
I20260809 19:20:55.145490 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9fa2815ee191423798ecf98c932837c9" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e11237ba54b81833627d0bf2dcb48"
I20260809 19:20:55.145563 11468 raft_consensus.cc:3037] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.145565 11671 raft_consensus.cc:3037] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.146198 11468 raft_consensus.cc:2454] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 1.
I20260809 19:20:55.146307 11671 raft_consensus.cc:2454] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 1.
I20260809 19:20:55.146356 11769 leader_election.cc:302] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc82b16658ea4335b82ddbf9d5567ba8, f80e11237ba54b81833627d0bf2dcb48; no voters:
I20260809 19:20:55.146447 11975 raft_consensus.cc:2781] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:55.146586 11975 raft_consensus.cc:686] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 LEADER]: Becoming Leader. State: Replica: cc82b16658ea4335b82ddbf9d5567ba8, State: Running, Role: LEADER
I20260809 19:20:55.146677 11975 consensus_queue.cc:237] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } }
I20260809 19:20:55.147528 11247 catalog_manager.cc:5495] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc82b16658ea4335b82ddbf9d5567ba8 (127.9.91.67). New cstate: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } health_report { overall_health: UNKNOWN } } }
I20260809 19:20:55.188506 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.186284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.9.91.67:41523)
0809 19:20:55.186343 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.186345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.186359 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:20:55.186440 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.186441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.186462 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.186510 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:20:55.186634 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.187129 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.187839 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.188016 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.188057 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.188216 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:20:55.188311 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:20:55.188362 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.188388 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.188407 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:20:55.190377 11468 raft_consensus.cc:1261] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Refusing update from remote peer cc82b16658ea4335b82ddbf9d5567ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:55.190379 11671 raft_consensus.cc:1261] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Refusing update from remote peer cc82b16658ea4335b82ddbf9d5567ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:55.190721 11979 consensus_queue.cc:1035] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:55.190861 11975 consensus_queue.cc:1035] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:55.193349 11986 mvcc.cc:205] Tried to move back new op lower bound from 7316698133257297920 to 7316698133081329664. Current Snapshot: MvccSnapshot[applied={T|T < 7316698133257297920}]
I20260809 19:20:55.193683 11984 mvcc.cc:205] Tried to move back new op lower bound from 7316698133257297920 to 7316698133081329664. Current Snapshot: MvccSnapshot[applied={T|T < 7316698133257297920}]
I20260809 19:20:55.196660 11247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:20:55.196782 11247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:20:55.197621 11247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c: Sending DeleteTablet for 3 replicas of tablet 9fa2815ee191423798ecf98c932837c9
I20260809 19:20:55.197978 11839 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fa2815ee191423798ecf98c932837c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:55 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:55080
I20260809 19:20:55.198021 11644 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fa2815ee191423798ecf98c932837c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:55 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53528
I20260809 19:20:55.198228 11999 tablet_replica.cc:330] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: stopping tablet replica
I20260809 19:20:55.198249 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fa2815ee191423798ecf98c932837c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:55 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:58594
I20260809 19:20:55.198365 11999 raft_consensus.cc:2229] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:20:55.198482 11999 raft_consensus.cc:2258] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:55.198535 11247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:20:55.198685 11247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:55.198685 12002 tablet_replica.cc:330] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: stopping tablet replica
I20260809 19:20:55.198814 12002 raft_consensus.cc:2229] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:20:55.198936 12002 raft_consensus.cc:2258] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:55.198459 12000 tablet_replica.cc:330] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: stopping tablet replica
I20260809 19:20:55.199057 12000 raft_consensus.cc:2229] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:20:55.199230 12000 raft_consensus.cc:2258] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:55.199342 11999 ts_tablet_manager.cc:1905] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:55.199649 12000 ts_tablet_manager.cc:1905] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:55.200119 12002 ts_tablet_manager.cc:1905] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:55.200134 11644 tablet_service.cc:1467] Processing CreateTablet for tablet c235e5d6a2354f2fb9cca936a8451534 (DEFAULT_TABLE table=test-table [id=15e4910af7464f1599c44e796c93f08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:55.200241 11644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c235e5d6a2354f2fb9cca936a8451534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:55.201386 12000 ts_tablet_manager.cc:1918] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:20:55.201452 12000 log.cc:1196] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/9fa2815ee191423798ecf98c932837c9
I20260809 19:20:55.201675 11839 tablet_service.cc:1467] Processing CreateTablet for tablet c235e5d6a2354f2fb9cca936a8451534 (DEFAULT_TABLE table=test-table [id=15e4910af7464f1599c44e796c93f08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:55.201759 12000 ts_tablet_manager.cc:1939] T 9fa2815ee191423798ecf98c932837c9 P 73f142ab1f0245c78e33b0513bc6b047: Deleting consensus metadata
I20260809 19:20:55.201777 12002 ts_tablet_manager.cc:1918] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:20:55.201838 12002 log.cc:1196] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/9fa2815ee191423798ecf98c932837c9
I20260809 19:20:55.201839 11967 tablet_bootstrap.cc:492] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: Bootstrap starting.
I20260809 19:20:55.202077 12002 ts_tablet_manager.cc:1939] T 9fa2815ee191423798ecf98c932837c9 P f80e11237ba54b81833627d0bf2dcb48: Deleting consensus metadata
I20260809 19:20:55.202240 11967 tablet_bootstrap.cc:654] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:55.202720 11224 catalog_manager.cc:4841] TS 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813): tablet 9fa2815ee191423798ecf98c932837c9 (table test-table [id=d4341865fcdc4ba7ab694b5f0929eb6e]) successfully deleted
I20260809 19:20:55.202841 11227 catalog_manager.cc:4841] TS f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697): tablet 9fa2815ee191423798ecf98c932837c9 (table test-table [id=d4341865fcdc4ba7ab694b5f0929eb6e]) successfully deleted
I20260809 19:20:55.202961 11967 tablet_bootstrap.cc:492] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: No bootstrap required, opened a new log
I20260809 19:20:55.203001 11967 ts_tablet_manager.cc:1397] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:20:55.203104 11443 tablet_service.cc:1467] Processing CreateTablet for tablet c235e5d6a2354f2fb9cca936a8451534 (DEFAULT_TABLE table=test-table [id=15e4910af7464f1599c44e796c93f08a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:55.203223 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c235e5d6a2354f2fb9cca936a8451534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:55.203310 11967 raft_consensus.cc:348] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.203372 11967 raft_consensus.cc:374] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:55.203397 11967 raft_consensus.cc:729] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f142ab1f0245c78e33b0513bc6b047, State: Initialized, Role: FOLLOWER
I20260809 19:20:55.203433 11967 consensus_queue.cc:260] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.203517 11967 ts_tablet_manager.cc:1428] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:55.204342 11966 tablet_bootstrap.cc:492] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: Bootstrap starting.
I20260809 19:20:55.204695 11966 tablet_bootstrap.cc:654] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:55.205268 11966 tablet_bootstrap.cc:492] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: No bootstrap required, opened a new log
I20260809 19:20:55.205313 11966 ts_tablet_manager.cc:1397] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:55.205452 11966 raft_consensus.cc:348] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.205520 11966 raft_consensus.cc:374] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:55.205540 11966 raft_consensus.cc:729] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80e11237ba54b81833627d0bf2dcb48, State: Initialized, Role: FOLLOWER
I20260809 19:20:55.205791 11966 consensus_queue.cc:260] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.205897 11966 ts_tablet_manager.cc:1428] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:20:55.207016 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c235e5d6a2354f2fb9cca936a8451534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:55.208279 11963 tablet_bootstrap.cc:492] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: Bootstrap starting.
I20260809 19:20:55.208426 11999 ts_tablet_manager.cc:1918] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:20:55.208498 11999 log.cc:1196] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/9fa2815ee191423798ecf98c932837c9
I20260809 19:20:55.208731 11999 ts_tablet_manager.cc:1939] T 9fa2815ee191423798ecf98c932837c9 P cc82b16658ea4335b82ddbf9d5567ba8: Deleting consensus metadata
I20260809 19:20:55.208783 11963 tablet_bootstrap.cc:654] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:55.209281 11224 catalog_manager.cc:4841] TS cc82b16658ea4335b82ddbf9d5567ba8 (127.9.91.67:41523): tablet 9fa2815ee191423798ecf98c932837c9 (table test-table [id=d4341865fcdc4ba7ab694b5f0929eb6e]) successfully deleted
I20260809 19:20:55.209520 11963 tablet_bootstrap.cc:492] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: No bootstrap required, opened a new log
I20260809 19:20:55.209563 11963 ts_tablet_manager.cc:1397] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:55.209694 11963 raft_consensus.cc:348] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.209750 11963 raft_consensus.cc:374] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:55.209769 11963 raft_consensus.cc:729] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc82b16658ea4335b82ddbf9d5567ba8, State: Initialized, Role: FOLLOWER
I20260809 19:20:55.209810 11963 consensus_queue.cc:260] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.209893 11963 ts_tablet_manager.cc:1428] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:55.336269 11990 raft_consensus.cc:482] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:55.336354 11990 raft_consensus.cc:504] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.336506 11990 leader_election.cc:288] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697), 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813)
I20260809 19:20:55.336689 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c235e5d6a2354f2fb9cca936a8451534" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e11237ba54b81833627d0bf2dcb48" is_pre_election: true
I20260809 19:20:55.336699 11671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c235e5d6a2354f2fb9cca936a8451534" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f142ab1f0245c78e33b0513bc6b047" is_pre_election: true
I20260809 19:20:55.336784 11671 raft_consensus.cc:2454] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 0.
I20260809 19:20:55.336789 11468 raft_consensus.cc:2454] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 0.
I20260809 19:20:55.336917 11767 leader_election.cc:302] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73f142ab1f0245c78e33b0513bc6b047, cc82b16658ea4335b82ddbf9d5567ba8; no voters:
I20260809 19:20:55.337033 11990 raft_consensus.cc:2781] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:55.337082 11990 raft_consensus.cc:482] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:55.337096 11990 raft_consensus.cc:3037] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.337723 11990 raft_consensus.cc:504] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.337842 11990 leader_election.cc:288] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 election: Requested vote from peers f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697), 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813)
I20260809 19:20:55.337994 11671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c235e5d6a2354f2fb9cca936a8451534" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f142ab1f0245c78e33b0513bc6b047"
I20260809 19:20:55.337982 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c235e5d6a2354f2fb9cca936a8451534" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e11237ba54b81833627d0bf2dcb48"
I20260809 19:20:55.338059 11671 raft_consensus.cc:3037] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.338076 11468 raft_consensus.cc:3037] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.338814 11671 raft_consensus.cc:2454] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 1.
I20260809 19:20:55.338907 11468 raft_consensus.cc:2454] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 1.
I20260809 19:20:55.338989 11767 leader_election.cc:302] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73f142ab1f0245c78e33b0513bc6b047, cc82b16658ea4335b82ddbf9d5567ba8; no voters:
I20260809 19:20:55.339088 11990 raft_consensus.cc:2781] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:55.339150 11990 raft_consensus.cc:686] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 LEADER]: Becoming Leader. State: Replica: cc82b16658ea4335b82ddbf9d5567ba8, State: Running, Role: LEADER
I20260809 19:20:55.339205 11990 consensus_queue.cc:237] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.339749 11247 catalog_manager.cc:5495] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc82b16658ea4335b82ddbf9d5567ba8 (127.9.91.67). New cstate: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } health_report { overall_health: HEALTHY } } }
Aug 09 19:20:55 dist-test-slave-1vmg krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:55.374308 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.367624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53922 (local address 127.9.91.126:34731)
0809 19:20:55.367789 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.367791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.367801 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:20:55.367873 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.367873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.367886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.367956 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:55.368427 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.368935 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.369611 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.369800 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.369857 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.371873 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:55.371875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:55.371875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:55.371882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:55.373330 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:55.373705 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:55.373705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:55.373706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:55.373757 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:55.373901 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:55.373902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:55.373902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:55.373993 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:55.374056 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.374172 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.374239 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260809 19:20:55.374418 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.367536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34731 (local address 127.0.0.1:53922)
0809 19:20:55.367707 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:55.367728 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:55.367808 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:55.367964 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:55.367965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:55.368297 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:55.368345 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.368348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.369018 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.369019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.369535 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.369537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.369873 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.369874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.369923 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.370393 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:55.370397 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:55.371775 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:55.373410 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:55.373418 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:55.373420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:55.373638 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:55.373803 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:55.373803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:55.373804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:55.373859 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:55.374050 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:55.374054 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:55.374097 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:20:55.374155 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:20:55.374301 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":55,"thread_start_us":79,"threads_started":1}
I20260809 19:20:55.376308 11247 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53922:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:20:55.376502 11247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:55.377911 11443 tablet_service.cc:1467] Processing CreateTablet for tablet d8e4fb17c8ee4054b1612d0fd1cb5be2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e1138ab850a4857bcee090b5f8fef04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:20:55.377985 11839 tablet_service.cc:1467] Processing CreateTablet for tablet d8e4fb17c8ee4054b1612d0fd1cb5be2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e1138ab850a4857bcee090b5f8fef04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:20:55.377983 11644 tablet_service.cc:1467] Processing CreateTablet for tablet d8e4fb17c8ee4054b1612d0fd1cb5be2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e1138ab850a4857bcee090b5f8fef04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:20:55.378103 11443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e4fb17c8ee4054b1612d0fd1cb5be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:55.378100 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e4fb17c8ee4054b1612d0fd1cb5be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:55.378167 11644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8e4fb17c8ee4054b1612d0fd1cb5be2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:55.380541 11966 tablet_bootstrap.cc:492] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: Bootstrap starting.
I20260809 19:20:55.380911 11966 tablet_bootstrap.cc:654] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:55.381547 11966 tablet_bootstrap.cc:492] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: No bootstrap required, opened a new log
I20260809 19:20:55.381596 11966 ts_tablet_manager.cc:1397] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:55.381721 11966 raft_consensus.cc:348] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.381771 11966 raft_consensus.cc:374] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:55.381784 11966 raft_consensus.cc:729] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f80e11237ba54b81833627d0bf2dcb48, State: Initialized, Role: FOLLOWER
I20260809 19:20:55.381820 11966 consensus_queue.cc:260] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.381886 11966 ts_tablet_manager.cc:1428] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:55.381978 11976 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.382023 11976 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:55.382316 11963 tablet_bootstrap.cc:492] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: Bootstrap starting.
I20260809 19:20:55.382611 11963 tablet_bootstrap.cc:654] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:55.382747 11967 tablet_bootstrap.cc:492] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: Bootstrap starting.
I20260809 19:20:55.383173 11967 tablet_bootstrap.cc:654] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:55.383272 11963 tablet_bootstrap.cc:492] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: No bootstrap required, opened a new log
I20260809 19:20:55.383307 11963 ts_tablet_manager.cc:1397] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:55.383401 11963 raft_consensus.cc:348] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.383468 11963 raft_consensus.cc:374] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:55.383493 11963 raft_consensus.cc:729] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc82b16658ea4335b82ddbf9d5567ba8, State: Initialized, Role: FOLLOWER
I20260809 19:20:55.383538 11963 consensus_queue.cc:260] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.383643 11963 ts_tablet_manager.cc:1428] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:55.383734 11990 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.383770 11967 tablet_bootstrap.cc:492] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: No bootstrap required, opened a new log
I20260809 19:20:55.383777 11990 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:55.383813 11967 ts_tablet_manager.cc:1397] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:55.383958 11967 raft_consensus.cc:348] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.384013 11967 raft_consensus.cc:374] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:55.384030 11967 raft_consensus.cc:729] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f142ab1f0245c78e33b0513bc6b047, State: Initialized, Role: FOLLOWER
I20260809 19:20:55.384071 11967 consensus_queue.cc:260] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.384178 11967 ts_tablet_manager.cc:1428] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:55.384244 11977 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.384353 11977 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:55.577507 11990 raft_consensus.cc:482] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:55.577617 11990 raft_consensus.cc:504] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.577772 11990 leader_election.cc:288] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697), 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813)
I20260809 19:20:55.577971 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4fb17c8ee4054b1612d0fd1cb5be2" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e11237ba54b81833627d0bf2dcb48" is_pre_election: true
I20260809 19:20:55.577989 11671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4fb17c8ee4054b1612d0fd1cb5be2" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f142ab1f0245c78e33b0513bc6b047" is_pre_election: true
I20260809 19:20:55.578080 11671 raft_consensus.cc:2454] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 0.
I20260809 19:20:55.578078 11468 raft_consensus.cc:2454] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 0.
I20260809 19:20:55.578248 11767 leader_election.cc:302] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 73f142ab1f0245c78e33b0513bc6b047, cc82b16658ea4335b82ddbf9d5567ba8, f80e11237ba54b81833627d0bf2dcb48; no voters:
I20260809 19:20:55.578377 11990 raft_consensus.cc:2781] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:55.578430 11990 raft_consensus.cc:482] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:55.578449 11990 raft_consensus.cc:3037] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.579142 11990 raft_consensus.cc:504] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.579268 11990 leader_election.cc:288] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 election: Requested vote from peers f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697), 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813)
I20260809 19:20:55.579420 11671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4fb17c8ee4054b1612d0fd1cb5be2" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f142ab1f0245c78e33b0513bc6b047"
I20260809 19:20:55.579422 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8e4fb17c8ee4054b1612d0fd1cb5be2" candidate_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f80e11237ba54b81833627d0bf2dcb48"
I20260809 19:20:55.579490 11671 raft_consensus.cc:3037] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.579492 11468 raft_consensus.cc:3037] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:55.580247 11671 raft_consensus.cc:2454] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 1.
I20260809 19:20:55.580247 11468 raft_consensus.cc:2454] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc82b16658ea4335b82ddbf9d5567ba8 in term 1.
I20260809 19:20:55.580442 11769 leader_election.cc:302] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc82b16658ea4335b82ddbf9d5567ba8, f80e11237ba54b81833627d0bf2dcb48; no voters:
I20260809 19:20:55.580542 11990 raft_consensus.cc:2781] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:55.580603 11990 raft_consensus.cc:686] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 LEADER]: Becoming Leader. State: Replica: cc82b16658ea4335b82ddbf9d5567ba8, State: Running, Role: LEADER
I20260809 19:20:55.580657 11990 consensus_queue.cc:237] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } }
I20260809 19:20:55.580966 11979 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: TxnStatusTablet state changed. Reason: New leader cc82b16658ea4335b82ddbf9d5567ba8. Latest consensus state: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.581044 11979 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:20:55.581197 12027 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:20:55.581324 12027 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:20:55.581305 11246 catalog_manager.cc:5495] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc82b16658ea4335b82ddbf9d5567ba8 (127.9.91.67). New cstate: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } health_report { overall_health: HEALTHY } } }
I20260809 19:20:55.607014 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.604892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55090 (local address 127.9.91.67:41523)
0809 19:20:55.604957 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.604959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.604968 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:20:55.605009 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.605009 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.605020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.605064 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:20:55.605182 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.605642 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.606362 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.606556 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.606608 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.606767 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:20:55.606804 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:20:55.606848 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.606928 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.606949 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:20:55.607038 12020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.604818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:41523 (local address 127.0.0.1:55090)
0809 19:20:55.604892 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:55.604903 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:55.604949 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:55.605070 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:55.605071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:55.605072 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:20:55.605122 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.605124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.605714 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.605716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.606282 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.606283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.606617 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.606618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.606677 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.606693 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:20:55.606857 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:20:55.606865 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 19:20:55.606907 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 19:20:55.606974 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 19:20:55.607997 11671 raft_consensus.cc:1261] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Refusing update from remote peer cc82b16658ea4335b82ddbf9d5567ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:55.608022 11468 raft_consensus.cc:1261] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Refusing update from remote peer cc82b16658ea4335b82ddbf9d5567ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:55.608184 11990 consensus_queue.cc:1035] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:55.608256 11979 consensus_queue.cc:1035] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:55.608776 11976 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: TxnStatusTablet state changed. Reason: New leader cc82b16658ea4335b82ddbf9d5567ba8. Latest consensus state: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.608795 11977 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: TxnStatusTablet state changed. Reason: New leader cc82b16658ea4335b82ddbf9d5567ba8. Latest consensus state: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.608867 11976 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:55.608891 11977 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:55.609543 11983 mvcc.cc:205] Tried to move back new op lower bound from 7316698134969012224 to 7316698134858772480. Current Snapshot: MvccSnapshot[applied={T|T < 7316698134969012224}]
I20260809 19:20:55.609620 11975 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.609702 11975 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:20:55.609935 11977 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.610054 11977 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P 73f142ab1f0245c78e33b0513bc6b047: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:55.610752 11976 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.610836 11976 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P f80e11237ba54b81833627d0bf2dcb48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:55.611084 11979 tablet_replica.cc:439] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 } } peers { permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 } } peers { permanent_uuid: "cc82b16658ea4335b82ddbf9d5567ba8" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41523 } } }
I20260809 19:20:55.611167 11979 tablet_replica.cc:442] T d8e4fb17c8ee4054b1612d0fd1cb5be2 P cc82b16658ea4335b82ddbf9d5567ba8: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:20:55.615698 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.613290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:51835 (local address 127.9.91.67:41523)
0809 19:20:55.613327 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.613329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.613447 (+ 118us) server_negotiation.cc:407] Connection header received
0809 19:20:55.613516 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.613517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.613523 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.613564 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:20:55.613672 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.614076 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.615137 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.615310 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.615354 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.615519 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:20:55.615542 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:20:55.615574 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.615646 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.615657 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:20:55.615805 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.613218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:41523 (local address 127.9.91.67:51835)
0809 19:20:55.613379 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:55.613390 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:55.613456 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:55.613571 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:55.613572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:55.613573 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:20:55.613621 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.613622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.614136 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.614137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.615046 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:55.615048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.615369 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.615370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:55.615424 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.615440 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:20:55.615590 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:20:55.615591 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:20:55.615646 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:20:55.615690 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260809 19:20:55.617941 11468 raft_consensus.cc:1261] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Refusing update from remote peer cc82b16658ea4335b82ddbf9d5567ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:55.617995 11671 raft_consensus.cc:1261] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Refusing update from remote peer cc82b16658ea4335b82ddbf9d5567ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:55.618120 11979 consensus_queue.cc:1035] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f80e11237ba54b81833627d0bf2dcb48" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:55.618201 11975 consensus_queue.cc:1035] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f142ab1f0245c78e33b0513bc6b047" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:55.629637 11247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:20:55.629734 11247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:20:55.630329 11247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10b0538b497d4e9e82d69db7db8b947c: Sending DeleteTablet for 3 replicas of tablet c235e5d6a2354f2fb9cca936a8451534
I20260809 19:20:55.630591 11443 tablet_service.cc:1514] Processing DeleteTablet for tablet c235e5d6a2354f2fb9cca936a8451534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:55 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:58594
I20260809 19:20:55.630641 11839 tablet_service.cc:1514] Processing DeleteTablet for tablet c235e5d6a2354f2fb9cca936a8451534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:55 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:55080
I20260809 19:20:55.630735 12002 tablet_replica.cc:330] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: stopping tablet replica
I20260809 19:20:55.630743 11644 tablet_service.cc:1514] Processing DeleteTablet for tablet c235e5d6a2354f2fb9cca936a8451534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:55 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53528
I20260809 19:20:55.630791 12002 raft_consensus.cc:2229] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:20:55.630820 12002 raft_consensus.cc:2258] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:55.630820 12000 tablet_replica.cc:330] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: stopping tablet replica
I20260809 19:20:55.630861 12000 raft_consensus.cc:2229] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:20:55.630877 11999 tablet_replica.cc:330] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: stopping tablet replica
I20260809 19:20:55.630895 12000 raft_consensus.cc:2258] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:55.631031 11999 raft_consensus.cc:2229] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:20:55.631095 11999 raft_consensus.cc:2258] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:55.631234 12000 ts_tablet_manager.cc:1905] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:55.631489 11999 ts_tablet_manager.cc:1905] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:55.632943 12000 ts_tablet_manager.cc:1918] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:20:55.633018 12000 log.cc:1196] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/c235e5d6a2354f2fb9cca936a8451534
I20260809 19:20:55.633064 11999 ts_tablet_manager.cc:1918] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:20:55.633111 11999 log.cc:1196] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/c235e5d6a2354f2fb9cca936a8451534
I20260809 19:20:55.633246 12000 ts_tablet_manager.cc:1939] T c235e5d6a2354f2fb9cca936a8451534 P 73f142ab1f0245c78e33b0513bc6b047: Deleting consensus metadata
I20260809 19:20:55.633307 11999 ts_tablet_manager.cc:1939] T c235e5d6a2354f2fb9cca936a8451534 P cc82b16658ea4335b82ddbf9d5567ba8: Deleting consensus metadata
I20260809 19:20:55.633697 11224 catalog_manager.cc:4841] TS cc82b16658ea4335b82ddbf9d5567ba8 (127.9.91.67:41523): tablet c235e5d6a2354f2fb9cca936a8451534 (table test-table [id=15e4910af7464f1599c44e796c93f08a]) successfully deleted
I20260809 19:20:55.633767 11224 catalog_manager.cc:4841] TS 73f142ab1f0245c78e33b0513bc6b047 (127.9.91.66:36813): tablet c235e5d6a2354f2fb9cca936a8451534 (table test-table [id=15e4910af7464f1599c44e796c93f08a]) successfully deleted
I20260809 19:20:55.633761 12002 ts_tablet_manager.cc:1905] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:55.635326 12002 ts_tablet_manager.cc:1918] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:20:55.635392 12002 log.cc:1196] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/c235e5d6a2354f2fb9cca936a8451534
I20260809 19:20:55.635615 12002 ts_tablet_manager.cc:1939] T c235e5d6a2354f2fb9cca936a8451534 P f80e11237ba54b81833627d0bf2dcb48: Deleting consensus metadata
I20260809 19:20:55.635914 11227 catalog_manager.cc:4841] TS f80e11237ba54b81833627d0bf2dcb48 (127.9.91.65:38697): tablet c235e5d6a2354f2fb9cca936a8451534 (table test-table [id=15e4910af7464f1599c44e796c93f08a]) successfully deleted
Aug 09 19:20:55 dist-test-slave-1vmg krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303254, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:20:55.640367 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.631413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.9.91.65:38697)
0809 19:20:55.631450 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.631452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.631802 (+ 350us) server_negotiation.cc:407] Connection header received
0809 19:20:55.631884 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.631885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.631894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.631920 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:55.632851 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.633573 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.634581 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.634787 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.634841 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.637811 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:55.637820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:55.637820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:55.637827 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:55.639303 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:55.639612 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:55.639616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:55.639616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:55.639696 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:55.639875 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:55.639876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:55.639876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:55.640004 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:55.640129 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.640211 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.640285 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:20:55.640667 11530 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58600
I20260809 19:20:55.647032 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:55.641531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.9.91.126:34731)
0809 19:20:55.641574 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:55.641576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:55.641634 (+ 58us) server_negotiation.cc:407] Connection header received
0809 19:20:55.641683 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:55.641684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:55.641694 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:55.641745 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:55.642206 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.642672 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.643533 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:55.643742 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:55.643795 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:55.644774 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:55.644776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:55.644776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:55.644783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:55.646017 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:55.646317 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:55.646318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:55.646318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:55.646373 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:55.646535 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:55.646536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:55.646536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:55.646652 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:55.646751 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:55.646853 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:20:55.646951 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:20:55.647243 11247 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53926
I20260809 19:20:55.647842 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11354
I20260809 19:20:55.665988 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11548
I20260809 19:20:55.685959 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11743
I20260809 19:20:55.698206 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11195
2026-08-09T19:20:55Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2738 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:20:55 dist-test-slave-1vmg krb5kdc[12049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:20:55 dist-test-slave-1vmg krb5kdc[12049](info): set up 2 sockets
Aug 09 19:20:55 dist-test-slave-1vmg krb5kdc[12049](info): commencing operation
krb5kdc: starting...
W20260809 19:20:56.844961 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:20:56 dist-test-slave-1vmg krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:20:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:20:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:20:56.957464 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40077
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:40077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:57.037946 12066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:57.038158 12066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:57.038188 12066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:57.039322 12066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:20:57.039361 12066 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:20:57.039377 12066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:57.039394 12066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:57.039409 12066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:20:57.039433 12066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:20:57.040910 12066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:40077
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:57.041153 12066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:57.041353 12066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:57.045837 12076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.045915 12072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:57.046103 12066 server_base.cc:1034] running on GCE node
W20260809 19:20:57.045837 12074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:57.046391 12066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:57.046679 12066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:57.047811 12066 hybrid_clock.cc:648] HybridClock initialized: now 1786303257047797 us; error 29 us; skew 500 ppm
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:57.050115 12066 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:20:57.050501 12066 webserver.cc:460] Webserver started at http://127.9.91.126:46667/ using document root <none> and password file <none>
I20260809 19:20:57.050693 12066 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:57.050740 12066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:57.050859 12066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:57.051873 12066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "422aa146ea7d43d18ec2f60f6556e030"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "fae9660d632e6ed09071deb918170750"
server_key_iv: "afa58e868e0a5c5fc183461bb16a44bd"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.052191 12066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "422aa146ea7d43d18ec2f60f6556e030"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "fae9660d632e6ed09071deb918170750"
server_key_iv: "afa58e868e0a5c5fc183461bb16a44bd"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.053375 12066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:20:57.053989 12083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.054163 12066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:57.054263 12066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "422aa146ea7d43d18ec2f60f6556e030"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "fae9660d632e6ed09071deb918170750"
server_key_iv: "afa58e868e0a5c5fc183461bb16a44bd"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.054333 12066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:57.078406 12066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:57.080646 12066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:57.080787 12066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:57.085722 12066 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:40077
I20260809 19:20:57.085770 12187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:40077 every 8 connection(s)
I20260809 19:20:57.086089 12066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:20:57.086779 12188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:57.088984 12188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Bootstrap starting.
I20260809 19:20:57.089596 12188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:57.089906 12188 log.cc:824] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:57.090654 12188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: No bootstrap required, opened a new log
I20260809 19:20:57.092301 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 12066
I20260809 19:20:57.092386 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:20:57.092553 9581 external_mini_cluster.cc:1428] Setting key d0c34c27490444faba5bf493323d2d7a
I20260809 19:20:57.092782 12188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422aa146ea7d43d18ec2f60f6556e030" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40077 } }
I20260809 19:20:57.092933 12188 raft_consensus.cc:374] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:57.092972 12188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 422aa146ea7d43d18ec2f60f6556e030, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.093109 12188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422aa146ea7d43d18ec2f60f6556e030" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40077 } }
I20260809 19:20:57.093190 12188 raft_consensus.cc:388] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:20:57.093225 12188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:20:57.093274 12188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:57.093878 12188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422aa146ea7d43d18ec2f60f6556e030" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40077 } }
I20260809 19:20:57.093993 12188 leader_election.cc:302] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 422aa146ea7d43d18ec2f60f6556e030; no voters:
I20260809 19:20:57.094156 12188 leader_election.cc:288] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:20:57.094192 12193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:57.094322 12193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [term 1 LEADER]: Becoming Leader. State: Replica: 422aa146ea7d43d18ec2f60f6556e030, State: Running, Role: LEADER
I20260809 19:20:57.094400 12188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:20:57.094403 12193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422aa146ea7d43d18ec2f60f6556e030" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40077 } }
I20260809 19:20:57.095144 12196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 422aa146ea7d43d18ec2f60f6556e030. Latest consensus state: current_term: 1 leader_uuid: "422aa146ea7d43d18ec2f60f6556e030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422aa146ea7d43d18ec2f60f6556e030" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40077 } } }
I20260809 19:20:57.095214 12196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:57.095422 12194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "422aa146ea7d43d18ec2f60f6556e030" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "422aa146ea7d43d18ec2f60f6556e030" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40077 } } }
I20260809 19:20:57.095479 12194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030 [sys.catalog]: This master's current role is: LEADER
I20260809 19:20:57.095500 12207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:20:57.096024 12207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:20:57.097590 12207 catalog_manager.cc:1347] Generated new cluster ID: 422894b444ee4b17a7feef757585762f
I20260809 19:20:57.097651 12207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:57.102520 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.093046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.9.91.126:40077)
0809 19:20:57.093222 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.093232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.093271 (+ 39us) server_negotiation.cc:407] Connection header received
0809 19:20:57.093394 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.093401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.093447 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.093562 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.094348 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.095826 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.096689 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.097285 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.097382 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.099679 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.099683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.099683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.099694 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.101167 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.101482 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.101486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.101486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.101577 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.101760 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.101761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.101762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.101975 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.102114 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.102216 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.102347 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260809 19:20:57.104276 12207 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:20:57.104773 12207 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:20:57.113337 12207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Generated new TSK 0
I20260809 19:20:57.113477 12207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:20:57.141788 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40077
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:57.244412 12221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:57.244614 12221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:57.244649 12221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:57.245776 12221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:57.245817 12221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:57.245846 12221 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:20:57.245862 12221 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:20:57.245885 12221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:20:57.247375 12221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:40077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:57.247617 12221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:57.247825 12221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:57.252662 12229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.252671 12231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.252671 12228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:57.252954 12221 server_base.cc:1034] running on GCE node
I20260809 19:20:57.253134 12221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:57.253360 12221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:57.254513 12221 hybrid_clock.cc:648] HybridClock initialized: now 1786303257254496 us; error 34 us; skew 500 ppm
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:57.257191 12221 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:20:57.257611 12221 webserver.cc:460] Webserver started at http://127.9.91.65:41331/ using document root <none> and password file <none>
I20260809 19:20:57.257802 12221 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:57.257845 12221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:57.257941 12221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:57.259111 12221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "547f1a0a10a348398a8dabe39a2d137f"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "41071d52cfd8f6166339e68fbddb7da5"
server_key_iv: "4b024d239dc012784ede48f8937b31fe"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.259442 12221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "547f1a0a10a348398a8dabe39a2d137f"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "41071d52cfd8f6166339e68fbddb7da5"
server_key_iv: "4b024d239dc012784ede48f8937b31fe"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.260825 12221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:20:57.261509 12241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.261656 12221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:57.261734 12221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "547f1a0a10a348398a8dabe39a2d137f"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "41071d52cfd8f6166339e68fbddb7da5"
server_key_iv: "4b024d239dc012784ede48f8937b31fe"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.261799 12221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:57.276254 12221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:57.278192 12221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:57.278323 12221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:57.278839 12221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:57.278887 12221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.278909 12221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:57.278966 12221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:57.286064 12221 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:42143
I20260809 19:20:57.286162 12408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:42143 every 8 connection(s)
I20260809 19:20:57.286574 12221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:20:57.287108 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 12221
I20260809 19:20:57.287205 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:20:57.287377 9581 external_mini_cluster.cc:1428] Setting key 6b2d3778e5f2dc3c4913cca597f1578f
I20260809 19:20:57.289934 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.279340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:46181 (local address 127.9.91.126:40077)
0809 19:20:57.279406 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.279409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.279642 (+ 233us) server_negotiation.cc:407] Connection header received
0809 19:20:57.280310 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.280311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.280328 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.280390 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.281254 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.282020 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.283542 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.283827 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.283897 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.286508 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.286509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.286509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.286517 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.288530 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.288980 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.288980 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.288980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.289029 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.289265 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.289265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.289266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.289361 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.289403 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.289741 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.289844 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:20:57.290059 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.279305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40077 (local address 127.9.91.65:46181)
0809 19:20:57.279490 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.279541 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.280203 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.280474 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.280478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.280842 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:57.281145 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.281152 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.282130 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.282131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.283453 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.283456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.283916 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.283917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.284058 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.284589 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:57.284597 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:57.286416 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:57.288617 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.288625 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.288628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.288913 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.289079 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.289080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.289080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.289209 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.289409 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:57.289413 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:57.289570 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.289695 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.289873 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260809 19:20:57.290529 12410 heartbeater.cc:347] Connected to a master server at 127.9.91.126:40077
I20260809 19:20:57.290603 12410 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:57.290782 12410 heartbeater.cc:511] Master 127.9.91.126:40077 requested a full tablet report, sending...
I20260809 19:20:57.291342 12106 ts_manager.cc:194] Registered new tserver with Master: 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143)
I20260809 19:20:57.292071 12106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:46181
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
I20260809 19:20:57.297993 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.291670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:41091 (local address 127.9.91.126:40077)
0809 19:20:57.291710 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.291712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.291876 (+ 164us) server_negotiation.cc:407] Connection header received
0809 19:20:57.291968 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.291968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.291979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.292004 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.292391 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.293107 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.294251 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.294510 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.294567 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.295660 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.295662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.295662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.295668 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.296929 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.297325 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.297326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.297326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.297370 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.297558 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.297559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.297559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.297652 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.297697 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.297856 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.297911 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:20:57.298058 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.291612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40077 (local address 127.9.91.65:41091)
0809 19:20:57.291813 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.291824 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.291910 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.292063 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.292064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.292250 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:57.292308 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.292310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.293222 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.293223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.294147 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.294151 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.294585 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.294586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.294682 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.295125 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:57.295130 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:57.295571 (+ 441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:57.297016 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.297018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.297019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.297239 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.297414 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.297415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.297415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.297503 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.297707 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:57.297708 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:57.297761 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.297844 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.297954 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:20:57.314074 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40077
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:57.397498 12423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:57.397711 12423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:57.397744 12423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:57.398861 12423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:57.398902 12423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:57.398950 12423 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:20:57.398972 12423 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:20:57.399001 12423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:20:57.400466 12423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:40077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:57.400698 12423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:57.400916 12423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:57.405658 12434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.405661 12431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.405721 12432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:57.405872 12423 server_base.cc:1034] running on GCE node
I20260809 19:20:57.406059 12423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:57.406281 12423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:57.407418 12423 hybrid_clock.cc:648] HybridClock initialized: now 1786303257407403 us; error 32 us; skew 500 ppm
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:57.409811 12423 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:20:57.410211 12423 webserver.cc:460] Webserver started at http://127.9.91.66:44447/ using document root <none> and password file <none>
I20260809 19:20:57.410403 12423 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:57.410449 12423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:57.410575 12423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:57.411603 12423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "ee3f158105864ba496f7937a643f0109"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "58ac97d282f01719943e32b4e1c6dc14"
server_key_iv: "3b2ff58b016796fe6f0e22a7d50775e7"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.411942 12423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "ee3f158105864ba496f7937a643f0109"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "58ac97d282f01719943e32b4e1c6dc14"
server_key_iv: "3b2ff58b016796fe6f0e22a7d50775e7"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.413231 12423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:20:57.413877 12444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.414023 12423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:20:57.414108 12423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "ee3f158105864ba496f7937a643f0109"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "58ac97d282f01719943e32b4e1c6dc14"
server_key_iv: "3b2ff58b016796fe6f0e22a7d50775e7"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.414167 12423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:57.428439 12423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:57.430513 12423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:57.430655 12423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:57.431234 12423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:57.431277 12423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.431305 12423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:57.431326 12423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:57.438293 12423 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:44399
I20260809 19:20:57.438346 12622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:44399 every 8 connection(s)
I20260809 19:20:57.438710 12423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:20:57.440550 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.431680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:50131 (local address 127.9.91.126:40077)
0809 19:20:57.431736 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.431739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.431933 (+ 194us) server_negotiation.cc:407] Connection header received
0809 19:20:57.432421 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.432423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.432437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.432514 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.433211 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.433947 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.435289 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.435492 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.435549 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.437977 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.437979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.437979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.437988 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.439494 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.439853 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.439854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.439854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.439900 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.440069 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.440069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.440069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.440176 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.440220 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.440423 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.440487 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:20:57.440721 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.431654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40077 (local address 127.9.91.66:50131)
0809 19:20:57.431806 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.431847 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.432326 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.432585 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.432590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.432845 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:57.433109 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.433115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.434037 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.434039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.435204 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.435207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.435575 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.435576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.435699 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.436174 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:57.436184 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:57.437883 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:57.439569 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.439573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.439575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.439800 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.439944 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.439944 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.439945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.440027 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.440219 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:57.440221 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:57.440328 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.440404 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.440516 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260809 19:20:57.441172 12623 heartbeater.cc:347] Connected to a master server at 127.9.91.126:40077
I20260809 19:20:57.441252 12623 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:57.441428 12623 heartbeater.cc:511] Master 127.9.91.126:40077 requested a full tablet report, sending...
I20260809 19:20:57.441841 12109 ts_manager.cc:194] Registered new tserver with Master: ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399)
I20260809 19:20:57.442303 12109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:50131
I20260809 19:20:57.447170 12632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.442119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40077 (local address 127.9.91.66:60427)
0809 19:20:57.442263 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.442274 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.442339 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.442516 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.442517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.442700 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:57.442757 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.442759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.443441 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.443442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.443940 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.443942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.444268 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.444269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.444346 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.444736 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:57.444741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:57.445008 (+ 267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:57.446201 (+ 1193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.446202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.446203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.446360 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.446527 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.446528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.446528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.446585 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.446839 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:57.446840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:57.446888 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.446971 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.447073 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260809 19:20:57.447165 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.442170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:60427 (local address 127.9.91.126:40077)
0809 19:20:57.442205 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.442207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.442325 (+ 118us) server_negotiation.cc:407] Connection header received
0809 19:20:57.442419 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.442420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.442430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.442475 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.442831 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.443352 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.444010 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.444210 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.444263 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.445086 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.445087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.445087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.445094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.446131 (+ 1037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.446419 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.446420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.446421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.446475 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.446636 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.446637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.446637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.446779 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.446836 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.446996 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.447071 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:20:57.448660 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 12423
I20260809 19:20:57.448748 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:20:57.448938 9581 external_mini_cluster.cc:1428] Setting key 7286bdf8a8da3d33be14189ecbecf63e
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:20:57.473740 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40077
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:20:57.557066 12636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:20:57.557278 12636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:20:57.557313 12636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:20:57.558475 12636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:20:57.558517 12636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:20:57.558545 12636 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 19:20:57.558560 12636 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:20:57.558583 12636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:20:57.560101 12636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:40077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:20:57.560353 12636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:20:57.560564 12636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:20:57.565397 12642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.565407 12647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:20:57.565591 12643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:20:57.565830 12636 server_base.cc:1034] running on GCE node
I20260809 19:20:57.566025 12636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:20:57.566265 12636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:20:57.567400 12636 hybrid_clock.cc:648] HybridClock initialized: now 1786303257567389 us; error 24 us; skew 500 ppm
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:20:57.569799 12636 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:20:57.570282 12636 webserver.cc:460] Webserver started at http://127.9.91.67:42311/ using document root <none> and password file <none>
I20260809 19:20:57.570487 12636 fs_manager.cc:359] Metadata directory not provided
I20260809 19:20:57.570534 12636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:20:57.570665 12636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:20:57.571749 12636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "943be8d7ba0445fbaaecbf25e6c7dabb"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "21968e6ae2c3beaf8012e9f1c80d82ea"
server_key_iv: "92dce73641c508c81208c964f8908447"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.572088 12636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "943be8d7ba0445fbaaecbf25e6c7dabb"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "21968e6ae2c3beaf8012e9f1c80d82ea"
server_key_iv: "92dce73641c508c81208c964f8908447"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.573344 12636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:20:57.573979 12655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.574131 12636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:20:57.574229 12636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "943be8d7ba0445fbaaecbf25e6c7dabb"
format_stamp: "Formatted at 2026-08-09 19:20:57 on dist-test-slave-1vmg"
server_key: "21968e6ae2c3beaf8012e9f1c80d82ea"
server_key_iv: "92dce73641c508c81208c964f8908447"
server_key_version: "encryptionkey@0"
I20260809 19:20:57.574290 12636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:20:57.587996 12636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:20:57.589934 12636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:20:57.590073 12636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:20:57.590678 12636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:20:57.590726 12636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.590757 12636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:20:57.590781 12636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:57.597810 12636 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:44099
I20260809 19:20:57.597934 12825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:44099 every 8 connection(s)
I20260809 19:20:57.598234 12636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:20:57.599054 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 12636
I20260809 19:20:57.599141 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:20:57.599381 9581 external_mini_cluster.cc:1428] Setting key 0bbca440c8e99485aa38c3dbe227a8c0
I20260809 19:20:57.601094 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.591153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:60279 (local address 127.9.91.126:40077)
0809 19:20:57.591224 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.591227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.591442 (+ 215us) server_negotiation.cc:407] Connection header received
0809 19:20:57.591939 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.591940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.591954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.592036 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.592887 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.593621 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.595135 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.595450 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.595529 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.598189 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.598191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.598191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.598200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.599873 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.600349 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.600350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.600350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.600407 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.600605 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.600605 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.600606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.600719 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.600764 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.600967 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.601029 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:20:57.601284 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.591124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40077 (local address 127.9.91.67:60279)
0809 19:20:57.591310 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.591351 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.591848 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.592144 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.592148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.592482 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:57.592776 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.592782 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.593723 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.593724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.595024 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.595028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.595627 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.595628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.595794 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.596295 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:57.596305 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:57.598095 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:57.599963 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.599972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.599974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.600282 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.600457 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.600458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.600458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.600561 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.600761 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:57.600763 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:57.600874 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.600948 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.601088 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260809 19:20:57.601686 12827 heartbeater.cc:347] Connected to a master server at 127.9.91.126:40077
I20260809 19:20:57.601768 12827 heartbeater.cc:464] Registering TS with master...
I20260809 19:20:57.601949 12827 heartbeater.cc:511] Master 127.9.91.126:40077 requested a full tablet report, sending...
I20260809 19:20:57.602308 12106 ts_manager.cc:194] Registered new tserver with Master: 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099)
I20260809 19:20:57.602723 12106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:60279
I20260809 19:20:57.607620 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.602813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:56343 (local address 127.9.91.126:40077)
0809 19:20:57.602852 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.602854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.602966 (+ 112us) server_negotiation.cc:407] Connection header received
0809 19:20:57.603042 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.603043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.603053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.603097 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.603393 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.603859 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.604520 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.604714 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.604767 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.605567 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.605569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.605569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.605575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.606638 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.606945 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.606946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.606947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.607003 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.607167 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.607167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.607168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.607268 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.607322 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.607466 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.607538 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:20:57.607632 12836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.602758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40077 (local address 127.9.91.67:56343)
0809 19:20:57.602893 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.602904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.602991 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.603101 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.603101 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.603261 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:57.603327 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.603329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.603944 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.603944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.604443 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.604444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.604772 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.604773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.604852 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.605203 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:57.605209 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:57.605487 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:57.606713 (+ 1226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.606714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.606715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.606865 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.607058 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.607058 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.607059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.607116 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.607321 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:57.607321 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:57.607377 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.607437 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.607532 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260809 19:20:57.610982 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:57.633481 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.627165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.9.91.126:40077)
0809 19:20:57.627228 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.627229 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.627238 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:20:57.627271 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.627272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.627282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.627333 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.627890 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.628385 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.629149 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.629338 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.629393 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.631430 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.631431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.631431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.631439 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.632472 (+ 1033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.632777 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.632778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.632779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.632835 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.632995 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.632996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.632996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.633104 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.633195 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.633313 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.633390 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:20:57.634860 12106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:20:57.636437 12106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:57.642832 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.639085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52386 (local address 127.9.91.66:44399)
0809 19:20:57.639268 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.639273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.639282 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:20:57.639319 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.639321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.639335 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.639408 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:57.639584 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.640657 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.641901 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.642472 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.642537 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.642595 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.642693 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.642733 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260809 19:20:57.642834 12856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.638593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44399 (local address 127.0.0.1:52386)
0809 19:20:57.638997 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.639008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.639064 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.639478 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.639478 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.639480 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:57.639513 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.639514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.640748 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.640749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.641827 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.641829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.642549 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.642550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.642611 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.642615 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.642671 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.642734 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":27,"thread_start_us":103,"threads_started":1}
I20260809 19:20:57.643106 12861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.638587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:44099 (local address 127.0.0.1:49712)
0809 19:20:57.638884 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.638945 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.639019 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.639342 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.639345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.639347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:57.639422 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.639424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.640773 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.640774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.641846 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.641849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.642815 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.642816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.642870 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.642874 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.643008 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.643048 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":30,"thread_start_us":64,"threads_started":1}
I20260809 19:20:57.643188 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.638706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.9.91.67:44099)
0809 19:20:57.638865 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.638871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.639103 (+ 232us) server_negotiation.cc:407] Connection header received
0809 19:20:57.639148 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.639150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.639166 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.639237 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:57.639514 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.640672 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.641935 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.642673 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.642729 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.642782 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.643044 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.643094 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260809 19:20:57.642843 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.638642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42143 (local address 127.0.0.1:56316)
0809 19:20:57.639156 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.639164 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.639207 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.639456 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.639457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.639459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:57.639512 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.639514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.640360 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.640361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.641491 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.641494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.642497 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.642498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.642587 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.642603 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.642678 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.642748 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":26,"thread_start_us":61,"threads_started":1}
I20260809 19:20:57.643494 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.638867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.9.91.65:42143)
0809 19:20:57.639063 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.639068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.639259 (+ 191us) server_negotiation.cc:407] Connection header received
0809 19:20:57.639307 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.639309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.639324 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.639391 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:57.639587 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.640279 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.641572 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.642344 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.642407 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.642462 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.643378 (+ 916us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.643412 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260809 19:20:57.644065 12728 tablet_service.cc:1467] Processing CreateTablet for tablet 1536268db1df47609a273d42d55db6d2 (DEFAULT_TABLE table=test-table [id=e7a3084bec3b464e9018a6c1f3cac635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:57.644095 12320 tablet_service.cc:1467] Processing CreateTablet for tablet 1536268db1df47609a273d42d55db6d2 (DEFAULT_TABLE table=test-table [id=e7a3084bec3b464e9018a6c1f3cac635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:57.644240 12520 tablet_service.cc:1467] Processing CreateTablet for tablet 1536268db1df47609a273d42d55db6d2 (DEFAULT_TABLE table=test-table [id=e7a3084bec3b464e9018a6c1f3cac635]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:57.644405 12728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1536268db1df47609a273d42d55db6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:57.644471 12320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1536268db1df47609a273d42d55db6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:57.644533 12520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1536268db1df47609a273d42d55db6d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:57.646857 12866 tablet_bootstrap.cc:492] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Bootstrap starting.
I20260809 19:20:57.647699 12866 tablet_bootstrap.cc:654] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:57.647953 12868 tablet_bootstrap.cc:492] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Bootstrap starting.
I20260809 19:20:57.648028 12866 log.cc:824] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:57.648166 12867 tablet_bootstrap.cc:492] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Bootstrap starting.
I20260809 19:20:57.648737 12867 tablet_bootstrap.cc:654] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:57.648804 12866 tablet_bootstrap.cc:492] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: No bootstrap required, opened a new log
I20260809 19:20:57.648871 12866 ts_tablet_manager.cc:1397] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:20:57.649083 12867 log.cc:824] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:57.649760 12867 tablet_bootstrap.cc:492] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: No bootstrap required, opened a new log
I20260809 19:20:57.649837 12867 ts_tablet_manager.cc:1397] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:20:57.649922 12868 tablet_bootstrap.cc:654] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:57.650247 12868 log.cc:824] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:20:57.651019 12868 tablet_bootstrap.cc:492] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: No bootstrap required, opened a new log
I20260809 19:20:57.651089 12868 ts_tablet_manager.cc:1397] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:20:57.651160 12866 raft_consensus.cc:348] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.651317 12866 raft_consensus.cc:374] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:57.651358 12866 raft_consensus.cc:729] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943be8d7ba0445fbaaecbf25e6c7dabb, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.651467 12866 consensus_queue.cc:260] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.651533 12867 raft_consensus.cc:348] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.651697 12867 raft_consensus.cc:374] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:57.651741 12866 ts_tablet_manager.cc:1428] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:20:57.651744 12867 raft_consensus.cc:729] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3f158105864ba496f7937a643f0109, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.651857 12827 heartbeater.cc:503] Master 127.9.91.126:40077 was elected leader, sending a full tablet report...
I20260809 19:20:57.651880 12867 consensus_queue.cc:260] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.652122 12867 ts_tablet_manager.cc:1428] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:20:57.652168 12623 heartbeater.cc:503] Master 127.9.91.126:40077 was elected leader, sending a full tablet report...
I20260809 19:20:57.653185 12868 raft_consensus.cc:348] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.653298 12868 raft_consensus.cc:374] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:57.653318 12868 raft_consensus.cc:729] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547f1a0a10a348398a8dabe39a2d137f, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.653416 12868 consensus_queue.cc:260] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.653653 12868 ts_tablet_manager.cc:1428] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:20:57.653702 12410 heartbeater.cc:503] Master 127.9.91.126:40077 was elected leader, sending a full tablet report...
W20260809 19:20:57.689332 12624 tablet.cc:2369] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:20:57.787374 12411 tablet.cc:2369] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:57.799355 12874 raft_consensus.cc:482] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:57.799461 12874 raft_consensus.cc:504] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.799813 12874 leader_election.cc:288] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099), 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143)
I20260809 19:20:57.802516 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.799984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:44099 (local address 127.9.91.66:50833)
0809 19:20:57.800042 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.800054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.800095 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.800239 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.800240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.800242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:57.800328 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.800330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.800988 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.800989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.801775 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.801778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.802290 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.802291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.802364 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.802369 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.802412 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.802444 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":3}
I20260809 19:20:57.802564 12863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.800047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:50833 (local address 127.9.91.67:44099)
0809 19:20:57.800089 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.800092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.800103 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:20:57.800159 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.800160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.800171 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.800226 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:57.800394 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.800916 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.801851 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.802215 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.802272 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.802281 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.802426 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.802466 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:20:57.802582 12864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.800040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:54235 (local address 127.9.91.65:42143)
0809 19:20:57.800094 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.800096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.800258 (+ 162us) server_negotiation.cc:407] Connection header received
0809 19:20:57.800296 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.800297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.800309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.800360 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:20:57.800507 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.800956 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.801916 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.802275 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.802341 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.802351 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.802488 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.802524 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:20:57.802742 12877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.799984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42143 (local address 127.9.91.66:54235)
0809 19:20:57.800178 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.800188 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.800252 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.800378 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.800379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.800381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:20:57.800450 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.800452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.801031 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.801031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.801833 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.801836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.802352 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.802353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.802420 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.802424 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.802476 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.802512 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":13,"thread_start_us":124,"threads_started":1}
I20260809 19:20:57.802956 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1536268db1df47609a273d42d55db6d2" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" is_pre_election: true
I20260809 19:20:57.803064 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1536268db1df47609a273d42d55db6d2" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f" is_pre_election: true
I20260809 19:20:57.803108 12755 raft_consensus.cc:2454] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 0.
I20260809 19:20:57.803201 12348 raft_consensus.cc:2454] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 0.
I20260809 19:20:57.803324 12448 leader_election.cc:302] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943be8d7ba0445fbaaecbf25e6c7dabb, ee3f158105864ba496f7937a643f0109; no voters:
I20260809 19:20:57.803452 12874 raft_consensus.cc:2781] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:57.803524 12874 raft_consensus.cc:482] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:57.803542 12874 raft_consensus.cc:3037] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:57.804493 12874 raft_consensus.cc:504] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.804608 12874 leader_election.cc:288] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 election: Requested vote from peers 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099), 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143)
I20260809 19:20:57.804759 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1536268db1df47609a273d42d55db6d2" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f"
I20260809 19:20:57.804832 12348 raft_consensus.cc:3037] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:57.805495 12348 raft_consensus.cc:2454] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 1.
I20260809 19:20:57.805599 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1536268db1df47609a273d42d55db6d2" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb"
I20260809 19:20:57.805661 12446 leader_election.cc:302] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 547f1a0a10a348398a8dabe39a2d137f, ee3f158105864ba496f7937a643f0109; no voters:
I20260809 19:20:57.805681 12755 raft_consensus.cc:3037] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:57.805748 12874 raft_consensus.cc:2781] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:57.805879 12874 raft_consensus.cc:686] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 1 LEADER]: Becoming Leader. State: Replica: ee3f158105864ba496f7937a643f0109, State: Running, Role: LEADER
I20260809 19:20:57.805989 12874 consensus_queue.cc:237] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } }
I20260809 19:20:57.806542 12755 raft_consensus.cc:2454] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 1.
I20260809 19:20:57.806815 12106 catalog_manager.cc:5495] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee3f158105864ba496f7937a643f0109 (127.9.91.66). New cstate: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } health_report { overall_health: HEALTHY } } }
Aug 09 19:20:57 dist-test-slave-1vmg krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303257, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:20:57.841322 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.835509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.9.91.126:40077)
0809 19:20:57.835573 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:57.835574 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:57.835633 (+ 59us) server_negotiation.cc:407] Connection header received
0809 19:20:57.835687 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:57.835688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:57.835696 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:57.835799 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:20:57.836175 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.836631 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.837222 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.837386 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.837424 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.839236 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:20:57.839237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:20:57.839237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:20:57.839243 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:20:57.840472 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.840787 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.840787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.840788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.840836 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:20:57.840964 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:20:57.840965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:20:57.840965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:20:57.841050 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:20:57.841098 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:57.841194 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:20:57.841261 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":42}
I20260809 19:20:57.841418 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:57.835449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40077 (local address 127.0.0.1:56828)
0809 19:20:57.835576 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:57.835585 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:57.835647 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:57.835806 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:57.835806 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:57.836077 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:20:57.836119 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.836121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.836684 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.836684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.837174 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:57.837175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:57.837429 (+ 254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:57.837430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:57.837464 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:57.837894 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:20:57.837898 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:20:57.839157 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:20:57.840528 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.840534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.840535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.840736 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.840878 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:20:57.840879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:20:57.840879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:20:57.840927 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:20:57.841091 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:20:57.841095 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:20:57.841133 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 19:20:57.841177 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:20:57.841308 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":48,"thread_start_us":59,"threads_started":1}
I20260809 19:20:57.843108 12106 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:56828:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:20:57.843276 12106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:57.844506 12728 tablet_service.cc:1467] Processing CreateTablet for tablet a863026717804b1383554b5fe06ffc60 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c4facca6eaa4d39ba2b28aebeb2966c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:20:57.844527 12320 tablet_service.cc:1467] Processing CreateTablet for tablet a863026717804b1383554b5fe06ffc60 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c4facca6eaa4d39ba2b28aebeb2966c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:20:57.844524 12520 tablet_service.cc:1467] Processing CreateTablet for tablet a863026717804b1383554b5fe06ffc60 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c4facca6eaa4d39ba2b28aebeb2966c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:20:57.844647 12728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a863026717804b1383554b5fe06ffc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:57.844666 12320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a863026717804b1383554b5fe06ffc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:57.844666 12520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a863026717804b1383554b5fe06ffc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:57.845620 12866 tablet_bootstrap.cc:492] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: Bootstrap starting.
I20260809 19:20:57.846087 12868 tablet_bootstrap.cc:492] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: Bootstrap starting.
I20260809 19:20:57.846117 12866 tablet_bootstrap.cc:654] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:57.846463 12868 tablet_bootstrap.cc:654] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:57.846800 12866 tablet_bootstrap.cc:492] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: No bootstrap required, opened a new log
I20260809 19:20:57.846801 12867 tablet_bootstrap.cc:492] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: Bootstrap starting.
I20260809 19:20:57.846856 12866 ts_tablet_manager.cc:1397] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:20:57.847044 12866 raft_consensus.cc:348] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:57.847150 12866 raft_consensus.cc:374] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:57.847172 12866 raft_consensus.cc:729] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943be8d7ba0445fbaaecbf25e6c7dabb, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.847218 12866 consensus_queue.cc:260] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:57.847249 12867 tablet_bootstrap.cc:654] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:57.847321 12866 ts_tablet_manager.cc:1428] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.847393 12873 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:57.847493 12873 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:57.847813 12868 tablet_bootstrap.cc:492] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: No bootstrap required, opened a new log
I20260809 19:20:57.847867 12868 ts_tablet_manager.cc:1397] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:20:57.847999 12867 tablet_bootstrap.cc:492] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: No bootstrap required, opened a new log
I20260809 19:20:57.848037 12867 ts_tablet_manager.cc:1397] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:20:57.848014 12868 raft_consensus.cc:348] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:57.848076 12868 raft_consensus.cc:374] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:57.848096 12868 raft_consensus.cc:729] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547f1a0a10a348398a8dabe39a2d137f, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.848138 12868 consensus_queue.cc:260] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:57.848176 12867 raft_consensus.cc:348] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:57.848224 12868 ts_tablet_manager.cc:1428] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.848230 12867 raft_consensus.cc:374] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:57.848248 12867 raft_consensus.cc:729] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3f158105864ba496f7937a643f0109, State: Initialized, Role: FOLLOWER
I20260809 19:20:57.848290 12867 consensus_queue.cc:260] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:57.848280 12876 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:57.848376 12867 ts_tablet_manager.cc:1428] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:57.848371 12876 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:57.848407 12874 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:57.848479 12874 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 19:20:57.848867 12828 tablet.cc:2369] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:20:58.068974 12874 raft_consensus.cc:482] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:58.069099 12874 raft_consensus.cc:504] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.069283 12874 leader_election.cc:288] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143), 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099)
I20260809 19:20:58.069526 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a863026717804b1383554b5fe06ffc60" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" is_pre_election: true
I20260809 19:20:58.069525 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a863026717804b1383554b5fe06ffc60" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f" is_pre_election: true
I20260809 19:20:58.069648 12755 raft_consensus.cc:2454] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 0.
I20260809 19:20:58.069648 12348 raft_consensus.cc:2454] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 0.
I20260809 19:20:58.069802 12448 leader_election.cc:302] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943be8d7ba0445fbaaecbf25e6c7dabb, ee3f158105864ba496f7937a643f0109; no voters:
I20260809 19:20:58.069900 12874 raft_consensus.cc:2781] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:58.069964 12874 raft_consensus.cc:482] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:58.069979 12874 raft_consensus.cc:3037] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:58.070704 12874 raft_consensus.cc:504] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.070829 12874 leader_election.cc:288] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 election: Requested vote from peers 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143), 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099)
I20260809 19:20:58.071062 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a863026717804b1383554b5fe06ffc60" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f"
I20260809 19:20:58.071066 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a863026717804b1383554b5fe06ffc60" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb"
I20260809 19:20:58.071147 12755 raft_consensus.cc:3037] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:58.071151 12348 raft_consensus.cc:3037] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:58.071754 12755 raft_consensus.cc:2454] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 1.
I20260809 19:20:58.071759 12348 raft_consensus.cc:2454] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 1.
I20260809 19:20:58.071913 12446 leader_election.cc:302] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 547f1a0a10a348398a8dabe39a2d137f, ee3f158105864ba496f7937a643f0109; no voters:
I20260809 19:20:58.072052 12874 raft_consensus.cc:2781] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:58.072106 12874 raft_consensus.cc:686] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [term 1 LEADER]: Becoming Leader. State: Replica: ee3f158105864ba496f7937a643f0109, State: Running, Role: LEADER
I20260809 19:20:58.072149 12874 consensus_queue.cc:237] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.072470 12878 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: TxnStatusTablet state changed. Reason: New leader ee3f158105864ba496f7937a643f0109. Latest consensus state: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:58.072564 12878 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:20:58.072788 12893 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:20:58.072758 12109 catalog_manager.cc:5495] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee3f158105864ba496f7937a643f0109 (127.9.91.66). New cstate: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } health_report { overall_health: UNKNOWN } } }
I20260809 19:20:58.072921 12893 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:20:58.130040 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:58.127567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.9.91.66:44399)
0809 19:20:58.127610 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:58.127612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:58.127683 (+ 71us) server_negotiation.cc:407] Connection header received
0809 19:20:58.127771 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:58.127772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:58.127793 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:58.127845 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:20:58.127972 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.128448 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.129170 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.129411 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.129474 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:58.129723 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:20:58.129810 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:20:58.129868 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:58.129948 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 19:20:58.129968 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:20:58.130098 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:58.127517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44399 (local address 127.0.0.1:52390)
0809 19:20:58.127636 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:58.127647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:58.127703 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:58.127859 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:58.127860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:58.127862 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:20:58.127914 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:58.127916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.128531 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.128533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:58.129088 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:58.129089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.129524 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.129526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:58.129610 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:58.129629 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:20:58.129877 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:20:58.129886 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 19:20:58.129944 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:20:58.129999 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 19:20:58.131772 12755 raft_consensus.cc:1261] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Refusing update from remote peer ee3f158105864ba496f7937a643f0109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:58.131768 12348 raft_consensus.cc:1261] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Refusing update from remote peer ee3f158105864ba496f7937a643f0109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:58.132113 12874 consensus_queue.cc:1035] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:58.132236 12878 consensus_queue.cc:1035] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:58.133358 12876 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: TxnStatusTablet state changed. Reason: New leader ee3f158105864ba496f7937a643f0109. Latest consensus state: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:58.133359 12873 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: TxnStatusTablet state changed. Reason: New leader ee3f158105864ba496f7937a643f0109. Latest consensus state: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:58.133452 12876 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:58.133452 12873 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:58.134622 12878 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:58.134714 12878 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:20:58.134634 12874 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:58.134753 12874 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P ee3f158105864ba496f7937a643f0109: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:20:58.134781 12900 mvcc.cc:205] Tried to move back new op lower bound from 7316698145305198592 to 7316698145063960576. Current Snapshot: MvccSnapshot[applied={T|T < 7316698145305198592}]
I20260809 19:20:58.134987 12902 mvcc.cc:205] Tried to move back new op lower bound from 7316698145305198592 to 7316698145063960576. Current Snapshot: MvccSnapshot[applied={T|T < 7316698145305198592}]
I20260809 19:20:58.135077 12876 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:58.135139 12876 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P 547f1a0a10a348398a8dabe39a2d137f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:58.135630 12873 tablet_replica.cc:439] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } } }
I20260809 19:20:58.135707 12873 tablet_replica.cc:442] T a863026717804b1383554b5fe06ffc60 P 943be8d7ba0445fbaaecbf25e6c7dabb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:20:58.136262 12901 mvcc.cc:205] Tried to move back new op lower bound from 7316698145305198592 to 7316698145063960576. Current Snapshot: MvccSnapshot[applied={T|T < 7316698145305198592 or (T in {7316698145305198592})}]
I20260809 19:20:58.139185 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:58.136795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52406 (local address 127.9.91.66:44399)
0809 19:20:58.136842 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:58.136844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:58.136872 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:20:58.136941 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:58.136941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:58.136948 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:58.136990 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:20:58.137116 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.137525 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.138596 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.138761 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.138801 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:58.138996 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:20:58.139025 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:20:58.139062 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:58.139127 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:20:58.139141 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:20:58.143627 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:58.141405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:48905 (local address 127.9.91.66:44399)
0809 19:20:58.141437 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:20:58.141438 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:20:58.141570 (+ 132us) server_negotiation.cc:407] Connection header received
0809 19:20:58.141622 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:20:58.141623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:20:58.141630 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:20:58.141668 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:20:58.141780 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.142225 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.142944 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.143134 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.143193 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:58.143436 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:20:58.143460 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:20:58.143495 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 19:20:58.143572 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:20:58.143586 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:20:58.143729 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:20:58.141369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44399 (local address 127.9.91.66:48905)
0809 19:20:58.141506 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:20:58.141520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:20:58.141573 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:20:58.141675 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:20:58.141676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:20:58.141677 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:20:58.141730 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:58.141731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.142282 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.142283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:58.142855 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:20:58.142857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:20:58.143217 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:20:58.143240 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:20:58.143325 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:20:58.143342 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:20:58.143517 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:20:58.143519 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:20:58.143570 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:20:58.143617 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260809 19:20:58.145825 12348 raft_consensus.cc:1261] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Refusing update from remote peer ee3f158105864ba496f7937a643f0109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:58.145825 12755 raft_consensus.cc:1261] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Refusing update from remote peer ee3f158105864ba496f7937a643f0109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:58.146003 12878 consensus_queue.cc:1035] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:58.146163 12878 consensus_queue.cc:1035] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:58.163221 12106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:20:58.163311 12106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:20:58.164036 12106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Sending DeleteTablet for 3 replicas of tablet 1536268db1df47609a273d42d55db6d2
I20260809 19:20:58.164291 12728 tablet_service.cc:1514] Processing DeleteTablet for tablet 1536268db1df47609a273d42d55db6d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:58 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:49712
I20260809 19:20:58.164309 12520 tablet_service.cc:1514] Processing DeleteTablet for tablet 1536268db1df47609a273d42d55db6d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:58 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:52386
I20260809 19:20:58.164563 12922 tablet_replica.cc:330] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: stopping tablet replica
I20260809 19:20:58.164633 12320 tablet_service.cc:1514] Processing DeleteTablet for tablet 1536268db1df47609a273d42d55db6d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:20:58 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:56316
I20260809 19:20:58.164748 12922 raft_consensus.cc:2229] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:20:58.164815 12922 raft_consensus.cc:2258] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:58.164793 12106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:20:58.164952 12106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:20:58.165114 12925 tablet_replica.cc:330] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: stopping tablet replica
I20260809 19:20:58.165213 12925 raft_consensus.cc:2229] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:20:58.165261 12925 raft_consensus.cc:2258] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:58.165256 12922 ts_tablet_manager.cc:1905] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:58.165412 12923 tablet_replica.cc:330] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: stopping tablet replica
I20260809 19:20:58.165500 12923 raft_consensus.cc:2229] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:20:58.165627 12923 raft_consensus.cc:2258] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:20:58.166092 12923 ts_tablet_manager.cc:1905] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:58.166326 12728 tablet_service.cc:1467] Processing CreateTablet for tablet 671d491d2136498ea89a3819a42e8e15 (DEFAULT_TABLE table=test-table [id=ef8465e563994ed085e4cc3d24220ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:58.166424 12728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d491d2136498ea89a3819a42e8e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:58.166973 12922 ts_tablet_manager.cc:1918] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:20:58.167037 12922 log.cc:1196] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/1536268db1df47609a273d42d55db6d2
I20260809 19:20:58.167263 12922 ts_tablet_manager.cc:1939] T 1536268db1df47609a273d42d55db6d2 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting consensus metadata
I20260809 19:20:58.167582 12866 tablet_bootstrap.cc:492] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: Bootstrap starting.
I20260809 19:20:58.167691 12923 ts_tablet_manager.cc:1918] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:20:58.167750 12923 log.cc:1196] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/1536268db1df47609a273d42d55db6d2
I20260809 19:20:58.167903 12866 tablet_bootstrap.cc:654] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:58.167972 12923 ts_tablet_manager.cc:1939] T 1536268db1df47609a273d42d55db6d2 P ee3f158105864ba496f7937a643f0109: Deleting consensus metadata
I20260809 19:20:58.168318 12087 catalog_manager.cc:4841] TS ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399): tablet 1536268db1df47609a273d42d55db6d2 (table test-table [id=e7a3084bec3b464e9018a6c1f3cac635]) successfully deleted
I20260809 19:20:58.168335 12086 catalog_manager.cc:4841] TS 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099): tablet 1536268db1df47609a273d42d55db6d2 (table test-table [id=e7a3084bec3b464e9018a6c1f3cac635]) successfully deleted
I20260809 19:20:58.168524 12866 tablet_bootstrap.cc:492] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: No bootstrap required, opened a new log
I20260809 19:20:58.168558 12866 ts_tablet_manager.cc:1397] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:20:58.168664 12866 raft_consensus.cc:348] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.168704 12866 raft_consensus.cc:374] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:58.168716 12866 raft_consensus.cc:729] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943be8d7ba0445fbaaecbf25e6c7dabb, State: Initialized, Role: FOLLOWER
I20260809 19:20:58.168751 12866 consensus_queue.cc:260] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.168828 12866 ts_tablet_manager.cc:1428] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:58.169123 12320 tablet_service.cc:1467] Processing CreateTablet for tablet 671d491d2136498ea89a3819a42e8e15 (DEFAULT_TABLE table=test-table [id=ef8465e563994ed085e4cc3d24220ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:58.169236 12320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d491d2136498ea89a3819a42e8e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:58.169260 12520 tablet_service.cc:1467] Processing CreateTablet for tablet 671d491d2136498ea89a3819a42e8e15 (DEFAULT_TABLE table=test-table [id=ef8465e563994ed085e4cc3d24220ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:20:58.169345 12520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 671d491d2136498ea89a3819a42e8e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:20:58.170099 12867 tablet_bootstrap.cc:492] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: Bootstrap starting.
I20260809 19:20:58.170478 12867 tablet_bootstrap.cc:654] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:58.170524 12868 tablet_bootstrap.cc:492] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: Bootstrap starting.
I20260809 19:20:58.170900 12868 tablet_bootstrap.cc:654] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: Neither blocks nor log segments found. Creating new log.
I20260809 19:20:58.171075 12867 tablet_bootstrap.cc:492] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: No bootstrap required, opened a new log
I20260809 19:20:58.171106 12867 ts_tablet_manager.cc:1397] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:20:58.171211 12867 raft_consensus.cc:348] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.171260 12867 raft_consensus.cc:374] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:58.171278 12867 raft_consensus.cc:729] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3f158105864ba496f7937a643f0109, State: Initialized, Role: FOLLOWER
I20260809 19:20:58.171316 12867 consensus_queue.cc:260] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.171396 12867 ts_tablet_manager.cc:1428] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:58.171548 12925 ts_tablet_manager.cc:1905] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:20:58.171545 12868 tablet_bootstrap.cc:492] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: No bootstrap required, opened a new log
I20260809 19:20:58.172137 12868 ts_tablet_manager.cc:1397] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:20:58.172254 12868 raft_consensus.cc:348] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.172305 12868 raft_consensus.cc:374] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:20:58.172317 12868 raft_consensus.cc:729] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547f1a0a10a348398a8dabe39a2d137f, State: Initialized, Role: FOLLOWER
I20260809 19:20:58.172350 12868 consensus_queue.cc:260] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.172423 12868 ts_tablet_manager.cc:1428] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:20:58.173096 12925 ts_tablet_manager.cc:1918] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:20:58.173171 12925 log.cc:1196] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/1536268db1df47609a273d42d55db6d2
I20260809 19:20:58.173404 12925 ts_tablet_manager.cc:1939] T 1536268db1df47609a273d42d55db6d2 P 547f1a0a10a348398a8dabe39a2d137f: Deleting consensus metadata
I20260809 19:20:58.173890 12084 catalog_manager.cc:4841] TS 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143): tablet 1536268db1df47609a273d42d55db6d2 (table test-table [id=e7a3084bec3b464e9018a6c1f3cac635]) successfully deleted
I20260809 19:20:58.480994 12878 raft_consensus.cc:482] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:20:58.481104 12878 raft_consensus.cc:504] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.481281 12878 leader_election.cc:288] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143), 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099)
I20260809 19:20:58.481544 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d491d2136498ea89a3819a42e8e15" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f" is_pre_election: true
I20260809 19:20:58.481544 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d491d2136498ea89a3819a42e8e15" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" is_pre_election: true
I20260809 19:20:58.481664 12755 raft_consensus.cc:2454] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 0.
I20260809 19:20:58.481664 12348 raft_consensus.cc:2454] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 0.
I20260809 19:20:58.481839 12446 leader_election.cc:302] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 547f1a0a10a348398a8dabe39a2d137f, ee3f158105864ba496f7937a643f0109; no voters:
I20260809 19:20:58.481951 12878 raft_consensus.cc:2781] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:20:58.481988 12878 raft_consensus.cc:482] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:20:58.482009 12878 raft_consensus.cc:3037] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:58.482752 12878 raft_consensus.cc:504] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.482870 12878 leader_election.cc:288] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 election: Requested vote from peers 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143), 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099)
I20260809 19:20:58.483043 12348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d491d2136498ea89a3819a42e8e15" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f"
I20260809 19:20:58.483079 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "671d491d2136498ea89a3819a42e8e15" candidate_uuid: "ee3f158105864ba496f7937a643f0109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb"
I20260809 19:20:58.483124 12348 raft_consensus.cc:3037] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:58.483147 12755 raft_consensus.cc:3037] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:20:58.483819 12755 raft_consensus.cc:2454] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 1.
I20260809 19:20:58.483861 12348 raft_consensus.cc:2454] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee3f158105864ba496f7937a643f0109 in term 1.
I20260809 19:20:58.483997 12448 leader_election.cc:302] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943be8d7ba0445fbaaecbf25e6c7dabb, ee3f158105864ba496f7937a643f0109; no voters:
I20260809 19:20:58.484090 12878 raft_consensus.cc:2781] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:20:58.484143 12878 raft_consensus.cc:686] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 1 LEADER]: Becoming Leader. State: Replica: ee3f158105864ba496f7937a643f0109, State: Running, Role: LEADER
I20260809 19:20:58.484187 12878 consensus_queue.cc:237] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:20:58.484793 12109 catalog_manager.cc:5495] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee3f158105864ba496f7937a643f0109 (127.9.91.66). New cstate: current_term: 1 leader_uuid: "ee3f158105864ba496f7937a643f0109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } health_report { overall_health: UNKNOWN } } }
I20260809 19:20:58.526036 12755 raft_consensus.cc:1261] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Refusing update from remote peer ee3f158105864ba496f7937a643f0109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:58.526187 12348 raft_consensus.cc:1261] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Refusing update from remote peer ee3f158105864ba496f7937a643f0109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:20:58.526233 12878 consensus_queue.cc:1035] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:20:58.526481 12878 consensus_queue.cc:1035] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:20:58.530511 12900 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 19:21:00.047302 12105 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:00.047407 12105 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:00.048226 12105 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Sending DeleteTablet for 3 replicas of tablet 671d491d2136498ea89a3819a42e8e15
I20260809 19:21:00.048476 12519 tablet_service.cc:1514] Processing DeleteTablet for tablet 671d491d2136498ea89a3819a42e8e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:00 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:52386
I20260809 19:21:00.048542 12728 tablet_service.cc:1514] Processing DeleteTablet for tablet 671d491d2136498ea89a3819a42e8e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:00 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:49712
I20260809 19:21:00.048720 12320 tablet_service.cc:1514] Processing DeleteTablet for tablet 671d491d2136498ea89a3819a42e8e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:00 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:56316
I20260809 19:21:00.048906 12964 tablet_replica.cc:330] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: stopping tablet replica
I20260809 19:21:00.048969 12964 raft_consensus.cc:2229] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:00.049012 12964 raft_consensus.cc:2258] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:00.049237 12964 ts_tablet_manager.cc:1905] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:00.050642 12964 ts_tablet_manager.cc:1918] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:21:00.050698 12964 log.cc:1196] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/671d491d2136498ea89a3819a42e8e15
I20260809 19:21:00.050902 12964 ts_tablet_manager.cc:1939] T 671d491d2136498ea89a3819a42e8e15 P 547f1a0a10a348398a8dabe39a2d137f: Deleting consensus metadata
I20260809 19:21:00.051250 12084 catalog_manager.cc:4841] TS 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143): tablet 671d491d2136498ea89a3819a42e8e15 (table test-table [id=ef8465e563994ed085e4cc3d24220ae3]) successfully deleted
I20260809 19:21:00.053471 12960 tablet_replica.cc:330] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: stopping tablet replica
I20260809 19:21:00.053550 12960 raft_consensus.cc:2229] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:00.053620 12960 raft_consensus.cc:2258] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:00.053745 12963 tablet_replica.cc:330] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: stopping tablet replica
I20260809 19:21:00.053812 12963 raft_consensus.cc:2229] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:00.053853 12963 raft_consensus.cc:2258] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:00.053853 12960 ts_tablet_manager.cc:1905] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:00.054077 12963 ts_tablet_manager.cc:1905] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:00.055454 12963 ts_tablet_manager.cc:1918] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:21:00.055507 12963 log.cc:1196] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/671d491d2136498ea89a3819a42e8e15
I20260809 19:21:00.055572 12960 ts_tablet_manager.cc:1918] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:21:00.055684 12960 log.cc:1196] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/671d491d2136498ea89a3819a42e8e15
I20260809 19:21:00.055735 12963 ts_tablet_manager.cc:1939] T 671d491d2136498ea89a3819a42e8e15 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting consensus metadata
I20260809 19:21:00.055968 12960 ts_tablet_manager.cc:1939] T 671d491d2136498ea89a3819a42e8e15 P ee3f158105864ba496f7937a643f0109: Deleting consensus metadata
I20260809 19:21:00.056100 12086 catalog_manager.cc:4841] TS 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099): tablet 671d491d2136498ea89a3819a42e8e15 (table test-table [id=ef8465e563994ed085e4cc3d24220ae3]) successfully deleted
I20260809 19:21:00.056288 12087 catalog_manager.cc:4841] TS ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399): tablet 671d491d2136498ea89a3819a42e8e15 (table test-table [id=ef8465e563994ed085e4cc3d24220ae3]) successfully deleted
Aug 09 19:21:00 dist-test-slave-1vmg krb5kdc[12049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303260, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:21:00 dist-test-slave-1vmg krb5kdc[12049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303260, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:00.077035 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.066085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.9.91.126:40077)
0809 19:21:00.066274 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.066276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.066300 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:21:00.066326 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.066326 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.066339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.066391 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:00.066983 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.067519 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.068267 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.068484 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.068547 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.074733 (+ 6186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:00.074734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:00.074734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:00.074742 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:00.076163 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:00.076429 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:00.076429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:00.076430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:00.076478 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:00.076599 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:00.076599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:00.076599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:00.076693 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:00.076764 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.076886 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.076954 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260809 19:21:00.078452 12105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:00.078665 12105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:00.080052 12320 tablet_service.cc:1467] Processing CreateTablet for tablet f6b8724c4a984796910040d82d632c99 (DEFAULT_TABLE table=test-table [id=21d0a8a229ea4e1db9d37f8c8960fedf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:00.080052 12519 tablet_service.cc:1467] Processing CreateTablet for tablet f6b8724c4a984796910040d82d632c99 (DEFAULT_TABLE table=test-table [id=21d0a8a229ea4e1db9d37f8c8960fedf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:00.080070 12728 tablet_service.cc:1467] Processing CreateTablet for tablet f6b8724c4a984796910040d82d632c99 (DEFAULT_TABLE table=test-table [id=21d0a8a229ea4e1db9d37f8c8960fedf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:00.080202 12320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6b8724c4a984796910040d82d632c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:00.080211 12728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6b8724c4a984796910040d82d632c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:00.080204 12519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6b8724c4a984796910040d82d632c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:00.081195 12975 tablet_bootstrap.cc:492] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: Bootstrap starting.
I20260809 19:21:00.081511 12975 tablet_bootstrap.cc:654] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:00.081749 12978 tablet_bootstrap.cc:492] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: Bootstrap starting.
I20260809 19:21:00.081766 12976 tablet_bootstrap.cc:492] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: Bootstrap starting.
I20260809 19:21:00.082129 12978 tablet_bootstrap.cc:654] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:00.082306 12976 tablet_bootstrap.cc:654] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:00.082938 12976 tablet_bootstrap.cc:492] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: No bootstrap required, opened a new log
I20260809 19:21:00.082996 12976 ts_tablet_manager.cc:1397] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:00.083165 12975 tablet_bootstrap.cc:492] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: No bootstrap required, opened a new log
I20260809 19:21:00.083154 12976 raft_consensus.cc:348] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.083250 12975 ts_tablet_manager.cc:1397] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:00.083257 12976 raft_consensus.cc:374] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:00.083287 12976 raft_consensus.cc:729] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547f1a0a10a348398a8dabe39a2d137f, State: Initialized, Role: FOLLOWER
I20260809 19:21:00.083360 12976 consensus_queue.cc:260] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.083408 12975 raft_consensus.cc:348] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.083504 12975 raft_consensus.cc:374] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:00.083534 12975 raft_consensus.cc:729] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943be8d7ba0445fbaaecbf25e6c7dabb, State: Initialized, Role: FOLLOWER
I20260809 19:21:00.083536 12976 ts_tablet_manager.cc:1428] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:00.083592 12975 consensus_queue.cc:260] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.083779 12975 ts_tablet_manager.cc:1428] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:00.084209 12978 tablet_bootstrap.cc:492] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: No bootstrap required, opened a new log
I20260809 19:21:00.084296 12978 ts_tablet_manager.cc:1397] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:21:00.084419 12978 raft_consensus.cc:348] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.084513 12978 raft_consensus.cc:374] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:00.084547 12978 raft_consensus.cc:729] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3f158105864ba496f7937a643f0109, State: Initialized, Role: FOLLOWER
I20260809 19:21:00.084590 12978 consensus_queue.cc:260] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.084686 12978 ts_tablet_manager.cc:1428] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:00.326056 12986 raft_consensus.cc:482] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:00.326150 12986 raft_consensus.cc:504] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.326500 12986 leader_election.cc:288] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399), 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099)
I20260809 19:21:00.330538 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.326686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:44099 (local address 127.9.91.65:47111)
0809 19:21:00.327005 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.327014 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.327066 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.327222 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.327223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.327224 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:00.327292 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.327293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.328171 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.328171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.329456 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.329458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.330222 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.330223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.330333 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.330338 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.330394 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.330441 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":22,"thread_start_us":141,"threads_started":1}
I20260809 19:21:00.330561 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.326778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:47111 (local address 127.9.91.67:44099)
0809 19:21:00.326941 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.326943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.327066 (+ 123us) server_negotiation.cc:407] Connection header received
0809 19:21:00.327134 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.327136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.327146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.327211 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:00.327356 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.328095 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.329562 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.330147 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.330212 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.330224 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.330429 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.330471 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260809 19:21:00.330693 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.326791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:33547 (local address 127.9.91.66:44399)
0809 19:21:00.326956 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.326958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.326970 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:21:00.327052 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.327053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.327064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.327133 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:00.327300 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.328076 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.329561 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.330138 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.330212 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.330224 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.330570 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.330625 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260809 19:21:00.330646 12989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.326683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44399 (local address 127.9.91.65:33547)
0809 19:21:00.326898 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.326930 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.326989 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.327139 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.327140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.327142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:00.327225 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.327227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.328159 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.328160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.329455 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.329458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.330220 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.330221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.330474 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.330478 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.330530 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.330565 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":54,"thread_start_us":137,"threads_started":1}
I20260809 19:21:00.331116 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b8724c4a984796910040d82d632c99" candidate_uuid: "547f1a0a10a348398a8dabe39a2d137f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" is_pre_election: true
I20260809 19:21:00.331140 12550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b8724c4a984796910040d82d632c99" candidate_uuid: "547f1a0a10a348398a8dabe39a2d137f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3f158105864ba496f7937a643f0109" is_pre_election: true
I20260809 19:21:00.331223 12755 raft_consensus.cc:2454] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547f1a0a10a348398a8dabe39a2d137f in term 0.
I20260809 19:21:00.331283 12550 raft_consensus.cc:2454] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 547f1a0a10a348398a8dabe39a2d137f in term 0.
I20260809 19:21:00.331398 12244 leader_election.cc:302] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 547f1a0a10a348398a8dabe39a2d137f, 943be8d7ba0445fbaaecbf25e6c7dabb; no voters:
I20260809 19:21:00.331581 12986 raft_consensus.cc:2781] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:00.331626 12986 raft_consensus.cc:482] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:00.331640 12986 raft_consensus.cc:3037] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:00.332312 12986 raft_consensus.cc:504] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.332427 12986 leader_election.cc:288] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [CANDIDATE]: Term 1 election: Requested vote from peers ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399), 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099)
I20260809 19:21:00.332572 12550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b8724c4a984796910040d82d632c99" candidate_uuid: "547f1a0a10a348398a8dabe39a2d137f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3f158105864ba496f7937a643f0109"
I20260809 19:21:00.332593 12755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6b8724c4a984796910040d82d632c99" candidate_uuid: "547f1a0a10a348398a8dabe39a2d137f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb"
I20260809 19:21:00.332653 12550 raft_consensus.cc:3037] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:00.332667 12755 raft_consensus.cc:3037] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:00.333292 12755 raft_consensus.cc:2454] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547f1a0a10a348398a8dabe39a2d137f in term 1.
I20260809 19:21:00.333387 12550 raft_consensus.cc:2454] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 547f1a0a10a348398a8dabe39a2d137f in term 1.
I20260809 19:21:00.333441 12244 leader_election.cc:302] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 547f1a0a10a348398a8dabe39a2d137f, 943be8d7ba0445fbaaecbf25e6c7dabb; no voters:
I20260809 19:21:00.333523 12986 raft_consensus.cc:2781] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:00.333653 12986 raft_consensus.cc:686] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 1 LEADER]: Becoming Leader. State: Replica: 547f1a0a10a348398a8dabe39a2d137f, State: Running, Role: LEADER
I20260809 19:21:00.333758 12986 consensus_queue.cc:237] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.334399 12109 catalog_manager.cc:5495] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f reported cstate change: term changed from 0 to 1, leader changed from <none> to 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65). New cstate: current_term: 1 leader_uuid: "547f1a0a10a348398a8dabe39a2d137f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:00.368774 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.366087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.9.91.65:42143)
0809 19:21:00.366252 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.366254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.366261 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:00.366299 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.366300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.366327 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.366403 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:00.366510 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.367051 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.368087 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.368318 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.368366 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.368498 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.368609 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.368656 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.368672 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.368687 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260809 19:21:00.372890 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.370980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35217 (local address 127.9.91.66:44399)
0809 19:21:00.371027 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.371028 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.371128 (+ 100us) server_negotiation.cc:407] Connection header received
0809 19:21:00.371172 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.371173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.371183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.371222 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:00.371308 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.371743 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.372410 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.372583 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.372623 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.372740 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.372763 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.372794 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.372823 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.372838 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:21:00.372912 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.370902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44399 (local address 127.9.91.65:35217)
0809 19:21:00.371069 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.371080 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.371133 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.371230 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.371230 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.371231 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:00.371276 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.371277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.371790 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.371791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.372357 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.372359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.372630 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.372630 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.372675 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.372687 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.372793 (+ 106us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.372794 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.372821 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.372851 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260809 19:21:00.374744 12755 raft_consensus.cc:1261] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Refusing update from remote peer 547f1a0a10a348398a8dabe39a2d137f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:00.374819 12550 raft_consensus.cc:1261] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Refusing update from remote peer 547f1a0a10a348398a8dabe39a2d137f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:00.375018 12992 consensus_queue.cc:1035] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [LEADER]: Connected to new peer: Peer: permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:00.375227 12992 consensus_queue.cc:1035] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:00.381635 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.379495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:52129 (local address 127.9.91.65:42143)
0809 19:21:00.379544 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.379546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.379617 (+ 71us) server_negotiation.cc:407] Connection header received
0809 19:21:00.379680 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.379681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.379690 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.379730 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:00.379851 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.380339 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.381033 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.381226 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.381276 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.381423 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.381452 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.381497 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.381558 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.381576 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:21:00.381685 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.379437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42143 (local address 127.9.91.66:52129)
0809 19:21:00.379557 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.379575 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.379630 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.379752 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.379753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.379754 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:00.379797 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.379798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.380408 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.380409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.380961 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.380963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.381283 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.381284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.381355 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.381359 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.381505 (+ 146us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.381509 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.381555 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.381604 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":79,"threads_started":1}
I20260809 19:21:00.388361 12105 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:00.388445 12105 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:00.388999 12105 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Sending DeleteTablet for 3 replicas of tablet f6b8724c4a984796910040d82d632c99
I20260809 19:21:00.389248 12728 tablet_service.cc:1514] Processing DeleteTablet for tablet f6b8724c4a984796910040d82d632c99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:00 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:49712
I20260809 19:21:00.389274 12320 tablet_service.cc:1514] Processing DeleteTablet for tablet f6b8724c4a984796910040d82d632c99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:00 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:56316
I20260809 19:21:00.389287 12519 tablet_service.cc:1514] Processing DeleteTablet for tablet f6b8724c4a984796910040d82d632c99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:00 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:52386
I20260809 19:21:00.389357 12964 tablet_replica.cc:330] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: stopping tablet replica
I20260809 19:21:00.389364 12963 tablet_replica.cc:330] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: stopping tablet replica
I20260809 19:21:00.389403 12964 raft_consensus.cc:2229] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:00.389413 12963 raft_consensus.cc:2229] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:00.389448 12963 raft_consensus.cc:2258] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:00.389479 12964 raft_consensus.cc:2258] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:00.389516 12960 tablet_replica.cc:330] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: stopping tablet replica
I20260809 19:21:00.389564 12960 raft_consensus.cc:2229] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:00.389591 12960 raft_consensus.cc:2258] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:00.389854 12964 ts_tablet_manager.cc:1905] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:00.389854 12963 ts_tablet_manager.cc:1905] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:00.391417 12960 ts_tablet_manager.cc:1905] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:00.391561 12105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 19:21:00.391645 12963 ts_tablet_manager.cc:1918] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
W20260809 19:21:00.391695 12105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:00.391705 12963 log.cc:1196] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/f6b8724c4a984796910040d82d632c99
I20260809 19:21:00.391916 12963 ts_tablet_manager.cc:1939] T f6b8724c4a984796910040d82d632c99 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting consensus metadata
I20260809 19:21:00.392813 12320 tablet_service.cc:1467] Processing CreateTablet for tablet f360e4d9ebe347bcadf5a9159c8461f4 (DEFAULT_TABLE table=test-table [id=2ec30a5739154b278080639ab640e563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:00.392925 12960 ts_tablet_manager.cc:1918] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:21:00.392967 12960 log.cc:1196] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/f6b8724c4a984796910040d82d632c99
I20260809 19:21:00.393121 12964 ts_tablet_manager.cc:1918] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:21:00.393165 12964 log.cc:1196] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/f6b8724c4a984796910040d82d632c99
I20260809 19:21:00.393167 12960 ts_tablet_manager.cc:1939] T f6b8724c4a984796910040d82d632c99 P ee3f158105864ba496f7937a643f0109: Deleting consensus metadata
I20260809 19:21:00.393339 12964 ts_tablet_manager.cc:1939] T f6b8724c4a984796910040d82d632c99 P 547f1a0a10a348398a8dabe39a2d137f: Deleting consensus metadata
I20260809 19:21:00.393474 12320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f360e4d9ebe347bcadf5a9159c8461f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:00.393460 12087 catalog_manager.cc:4841] TS ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399): tablet f6b8724c4a984796910040d82d632c99 (table test-table [id=21d0a8a229ea4e1db9d37f8c8960fedf]) successfully deleted
I20260809 19:21:00.393486 12086 catalog_manager.cc:4841] TS 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099): tablet f6b8724c4a984796910040d82d632c99 (table test-table [id=21d0a8a229ea4e1db9d37f8c8960fedf]) successfully deleted
I20260809 19:21:00.393591 12519 tablet_service.cc:1467] Processing CreateTablet for tablet f360e4d9ebe347bcadf5a9159c8461f4 (DEFAULT_TABLE table=test-table [id=2ec30a5739154b278080639ab640e563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:00.393671 12519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f360e4d9ebe347bcadf5a9159c8461f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:00.394747 12976 tablet_bootstrap.cc:492] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f: Bootstrap starting.
I20260809 19:21:00.394814 12978 tablet_bootstrap.cc:492] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: Bootstrap starting.
I20260809 19:21:00.394866 12728 tablet_service.cc:1467] Processing CreateTablet for tablet f360e4d9ebe347bcadf5a9159c8461f4 (DEFAULT_TABLE table=test-table [id=2ec30a5739154b278080639ab640e563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:00.394977 12728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f360e4d9ebe347bcadf5a9159c8461f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:00.395228 12978 tablet_bootstrap.cc:654] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:00.395262 12976 tablet_bootstrap.cc:654] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:00.395998 12976 tablet_bootstrap.cc:492] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f: No bootstrap required, opened a new log
I20260809 19:21:00.396042 12976 ts_tablet_manager.cc:1397] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:00.396142 12084 catalog_manager.cc:4841] TS 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143): tablet f6b8724c4a984796910040d82d632c99 (table test-table [id=21d0a8a229ea4e1db9d37f8c8960fedf]) successfully deleted
I20260809 19:21:00.396166 12975 tablet_bootstrap.cc:492] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: Bootstrap starting.
I20260809 19:21:00.396210 12976 raft_consensus.cc:348] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.396272 12976 raft_consensus.cc:374] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:00.396292 12976 raft_consensus.cc:729] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547f1a0a10a348398a8dabe39a2d137f, State: Initialized, Role: FOLLOWER
I20260809 19:21:00.396339 12976 consensus_queue.cc:260] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.396440 12976 ts_tablet_manager.cc:1428] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:00.396526 12978 tablet_bootstrap.cc:492] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: No bootstrap required, opened a new log
I20260809 19:21:00.396579 12978 ts_tablet_manager.cc:1397] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:00.396695 12978 raft_consensus.cc:348] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.396759 12978 raft_consensus.cc:374] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:00.396777 12978 raft_consensus.cc:729] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3f158105864ba496f7937a643f0109, State: Initialized, Role: FOLLOWER
I20260809 19:21:00.396771 12975 tablet_bootstrap.cc:654] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:00.396804 12978 consensus_queue.cc:260] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.396895 12978 ts_tablet_manager.cc:1428] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:00.397624 12975 tablet_bootstrap.cc:492] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: No bootstrap required, opened a new log
I20260809 19:21:00.397687 12975 ts_tablet_manager.cc:1397] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:00.397822 12975 raft_consensus.cc:348] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.397878 12975 raft_consensus.cc:374] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:00.397897 12975 raft_consensus.cc:729] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943be8d7ba0445fbaaecbf25e6c7dabb, State: Initialized, Role: FOLLOWER
I20260809 19:21:00.397941 12975 consensus_queue.cc:260] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.398020 12975 ts_tablet_manager.cc:1428] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:00.441267 12987 raft_consensus.cc:482] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:00.441404 12987 raft_consensus.cc:504] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.441749 12987 leader_election.cc:288] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143), ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399)
I20260809 19:21:00.444630 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.442088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:49449 (local address 127.9.91.66:44399)
0809 19:21:00.442148 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.442150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.442161 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:00.442249 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.442251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.442261 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.442311 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:00.442552 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.443061 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.443976 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.444328 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.444387 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.444400 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.444530 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.444565 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:21:00.444646 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.441908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44399 (local address 127.9.91.67:49449)
0809 19:21:00.442092 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.442104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.442159 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.442422 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.442423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.442426 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:00.442493 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.442495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.443132 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.443134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.443895 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.443897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.444400 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.444402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.444471 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.444475 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.444519 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.444557 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260809 19:21:00.445065 12550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f360e4d9ebe347bcadf5a9159c8461f4" candidate_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3f158105864ba496f7937a643f0109" is_pre_election: true
I20260809 19:21:00.445144 12550 raft_consensus.cc:2454] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 943be8d7ba0445fbaaecbf25e6c7dabb in term 0.
I20260809 19:21:00.445370 12662 leader_election.cc:302] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 943be8d7ba0445fbaaecbf25e6c7dabb, ee3f158105864ba496f7937a643f0109; no voters:
I20260809 19:21:00.445503 12987 raft_consensus.cc:2781] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:00.445540 12987 raft_consensus.cc:482] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:00.445559 12987 raft_consensus.cc:3037] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:00.445637 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.442074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:33043 (local address 127.9.91.65:42143)
0809 19:21:00.442121 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.442123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.442309 (+ 186us) server_negotiation.cc:407] Connection header received
0809 19:21:00.442419 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.442420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.442429 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.442482 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:00.442690 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.443391 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.444766 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.445283 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.445338 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.445350 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.445533 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.445578 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:21:00.445813 13018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.441923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42143 (local address 127.9.91.67:33043)
0809 19:21:00.442223 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.442234 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.442311 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.442512 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.442513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.442515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:00.442593 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.442595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.443541 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.443542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.444646 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.444649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.445367 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.445368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.445442 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.445447 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.445706 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.445746 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":96,"threads_started":1}
I20260809 19:21:00.446053 12346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f360e4d9ebe347bcadf5a9159c8461f4" candidate_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f" is_pre_election: true
I20260809 19:21:00.446159 12346 raft_consensus.cc:2454] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 943be8d7ba0445fbaaecbf25e6c7dabb in term 0.
I20260809 19:21:00.446434 12987 raft_consensus.cc:504] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.446554 12987 leader_election.cc:288] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [CANDIDATE]: Term 1 election: Requested vote from peers 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143), ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399)
I20260809 19:21:00.446743 12550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f360e4d9ebe347bcadf5a9159c8461f4" candidate_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee3f158105864ba496f7937a643f0109"
I20260809 19:21:00.446841 12550 raft_consensus.cc:3037] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:00.446983 12346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f360e4d9ebe347bcadf5a9159c8461f4" candidate_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "547f1a0a10a348398a8dabe39a2d137f"
I20260809 19:21:00.447052 12346 raft_consensus.cc:3037] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:00.447619 12346 raft_consensus.cc:2454] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 943be8d7ba0445fbaaecbf25e6c7dabb in term 1.
I20260809 19:21:00.447760 12550 raft_consensus.cc:2454] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 943be8d7ba0445fbaaecbf25e6c7dabb in term 1.
I20260809 19:21:00.447801 12657 leader_election.cc:302] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 547f1a0a10a348398a8dabe39a2d137f, 943be8d7ba0445fbaaecbf25e6c7dabb; no voters:
I20260809 19:21:00.447881 12987 raft_consensus.cc:2781] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:00.448027 12987 raft_consensus.cc:686] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 LEADER]: Becoming Leader. State: Replica: 943be8d7ba0445fbaaecbf25e6c7dabb, State: Running, Role: LEADER
I20260809 19:21:00.448102 12987 consensus_queue.cc:237] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } }
I20260809 19:21:00.448815 12109 catalog_manager.cc:5495] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb reported cstate change: term changed from 0 to 1, leader changed from <none> to 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67). New cstate: current_term: 1 leader_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "943be8d7ba0445fbaaecbf25e6c7dabb" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44099 } health_report { overall_health: HEALTHY } } }
I20260809 19:21:00.458063 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.455515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.9.91.67:44099)
0809 19:21:00.455552 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.455554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.455592 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:21:00.455680 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.455680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.455703 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.455761 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:00.455902 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.456350 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.457339 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.457521 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.457570 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.457741 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.457867 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.457920 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.457996 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.458013 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:00.463083 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.460509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:58237 (local address 127.9.91.66:44399)
0809 19:21:00.460553 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.460555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.460687 (+ 132us) server_negotiation.cc:407] Connection header received
0809 19:21:00.460741 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.460742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.460755 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.460807 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:00.460902 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.461574 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.462310 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.462601 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.462694 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.462820 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.462854 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.462936 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.462985 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.463010 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:21:00.463115 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.460333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44399 (local address 127.9.91.67:58237)
0809 19:21:00.460495 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.460505 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.460681 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.460815 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.460816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.460817 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:00.460863 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.460864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.461656 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.461657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.462222 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.462224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.462696 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.462696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.462737 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.462750 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.462930 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.462932 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.462969 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.463007 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260809 19:21:00.466769 12348 raft_consensus.cc:1261] T f360e4d9ebe347bcadf5a9159c8461f4 P 547f1a0a10a348398a8dabe39a2d137f [term 1 FOLLOWER]: Refusing update from remote peer 943be8d7ba0445fbaaecbf25e6c7dabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:00.467048 13019 consensus_queue.cc:1035] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "547f1a0a10a348398a8dabe39a2d137f" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:00.467090 12550 raft_consensus.cc:1261] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Refusing update from remote peer 943be8d7ba0445fbaaecbf25e6c7dabb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:00.467309 13019 consensus_queue.cc:1035] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee3f158105864ba496f7937a643f0109" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:00.474509 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.472238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:53251 (local address 127.9.91.67:44099)
0809 19:21:00.472273 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:00.472275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:00.472313 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:21:00.472387 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:00.472388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:00.472395 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:00.472441 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:00.472550 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.473174 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.473866 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.474069 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.474120 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.474270 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.474301 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.474349 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:00.474442 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 19:21:00.474461 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:21:00.474582 13009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:00.472194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:44099 (local address 127.9.91.66:53251)
0809 19:21:00.472263 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:00.472276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:00.472328 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:00.472456 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:00.472458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:00.472459 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:00.472497 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.472498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.473231 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.473233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.473785 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:00.473787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:00.474127 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:00.474128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:00.474192 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:00.474200 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:00.474361 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:00.474364 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:00.474438 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:21:00.474502 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 19:21:00.475054 12939 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 19:21:01.995129 12106 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:01.995229 12106 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:01.996050 12106 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 422aa146ea7d43d18ec2f60f6556e030: Sending DeleteTablet for 3 replicas of tablet f360e4d9ebe347bcadf5a9159c8461f4
I20260809 19:21:01.996304 12320 tablet_service.cc:1514] Processing DeleteTablet for tablet f360e4d9ebe347bcadf5a9159c8461f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:01 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:56316
I20260809 19:21:01.996327 12519 tablet_service.cc:1514] Processing DeleteTablet for tablet f360e4d9ebe347bcadf5a9159c8461f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:01 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:52386
I20260809 19:21:01.996369 12728 tablet_service.cc:1514] Processing DeleteTablet for tablet f360e4d9ebe347bcadf5a9159c8461f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:01 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:49712
I20260809 19:21:01.996430 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 12221
I20260809 19:21:02.001416 13061 tablet_replica.cc:330] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: stopping tablet replica
I20260809 19:21:02.001490 13061 raft_consensus.cc:2229] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:02.001646 13061 raft_consensus.cc:2258] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:02.001209 13058 tablet_replica.cc:330] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: stopping tablet replica
I20260809 19:21:02.002425 13058 raft_consensus.cc:2229] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:02.002476 13058 raft_consensus.cc:2258] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:02.002759 13061 ts_tablet_manager.cc:1905] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:02.002755 13058 ts_tablet_manager.cc:1905] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:02.004312 13058 ts_tablet_manager.cc:1918] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:21:02.004366 13058 log.cc:1196] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/f360e4d9ebe347bcadf5a9159c8461f4
I20260809 19:21:02.004383 13061 ts_tablet_manager.cc:1918] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 19:21:02.004472 13061 log.cc:1196] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/f360e4d9ebe347bcadf5a9159c8461f4
I20260809 19:21:02.004571 13058 ts_tablet_manager.cc:1939] T f360e4d9ebe347bcadf5a9159c8461f4 P ee3f158105864ba496f7937a643f0109: Deleting consensus metadata
I20260809 19:21:02.004743 13061 ts_tablet_manager.cc:1939] T f360e4d9ebe347bcadf5a9159c8461f4 P 943be8d7ba0445fbaaecbf25e6c7dabb: Deleting consensus metadata
I20260809 19:21:02.004880 12087 catalog_manager.cc:4841] TS ee3f158105864ba496f7937a643f0109 (127.9.91.66:44399): tablet f360e4d9ebe347bcadf5a9159c8461f4 (table test-table [id=2ec30a5739154b278080639ab640e563]) successfully deleted
I20260809 19:21:02.005019 12086 catalog_manager.cc:4841] TS 943be8d7ba0445fbaaecbf25e6c7dabb (127.9.91.67:44099): tablet f360e4d9ebe347bcadf5a9159c8461f4 (table test-table [id=2ec30a5739154b278080639ab640e563]) successfully deleted
W20260809 19:21:02.008503 12084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:42143 (error 108)
I20260809 19:21:02.008673 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 12423
I20260809 19:21:02.011112 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:02.010899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42143 (local address 127.0.0.1:56336)
0809 19:21:02.011026 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:02.011078 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:42143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":82,"threads_started":1}
W20260809 19:21:02.011230 12084 catalog_manager.cc:4584] TS 547f1a0a10a348398a8dabe39a2d137f (127.9.91.65:42143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f360e4d9ebe347bcadf5a9159c8461f4: Network error: Client connection negotiation failed: client connection to 127.9.91.65:42143: connect: Connection refused (error 111)
I20260809 19:21:02.016993 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 12636
I20260809 19:21:02.025460 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 12066
2026-08-09T19:21:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6328 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:02 dist-test-slave-1vmg krb5kdc[13067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:02 dist-test-slave-1vmg krb5kdc[13067](info): set up 2 sockets
Aug 09 19:21:02 dist-test-slave-1vmg krb5kdc[13067](info): commencing operation
krb5kdc: starting...
W20260809 19:21:03.201352 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 19:21:03.324779 9581 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.123s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 19:21:03.458699 9581 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.134s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 19:21:03.577647 9581 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.119s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/test-user.keytab.
W20260809 19:21:04.097357 9581 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.520s user 0.000s sys 0.002s
Aug 09 19:21:04 dist-test-slave-1vmg krb5kdc[13067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
W20260809 19:21:04.739598 9581 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.91.126: real 0.621s user 0.000s sys 0.003s
I20260809 19:21:04.740068 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38515
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:38515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:04.819836 13088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:04.820007 13088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:04.820029 13088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:04.821085 13088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:04.821117 13088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:04.821130 13088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:04.821141 13088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:04.821153 13088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:04.822511 13088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:38515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:04.822701 13088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:04.822860 13088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:04.827509 13094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:04.827620 13088 server_base.cc:1034] running on GCE node
W20260809 19:21:04.827518 13096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:04.827521 13093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:04.828158 13088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:04.828392 13088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:04.829516 13088 hybrid_clock.cc:648] HybridClock initialized: now 1786303264829503 us; error 33 us; skew 500 ppm
Aug 09 19:21:04 dist-test-slave-1vmg krb5kdc[13067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303264, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:04.831853 13088 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:04.832255 13088 webserver.cc:460] Webserver started at http://127.9.91.126:39007/ using document root <none> and password file <none>
I20260809 19:21:04.832440 13088 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:04.832487 13088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:04.832607 13088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:04.833608 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "1725160d154144cc860d738c94013877"
format_stamp: "Formatted at 2026-08-09 19:21:04 on dist-test-slave-1vmg"
server_key: "e1517cc83e56d94dbecee5d8075be6df"
server_key_iv: "d236f5a8d8ef45c85854162adc83c0c3"
server_key_version: "encryptionkey@0"
I20260809 19:21:04.833976 13088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "1725160d154144cc860d738c94013877"
format_stamp: "Formatted at 2026-08-09 19:21:04 on dist-test-slave-1vmg"
server_key: "e1517cc83e56d94dbecee5d8075be6df"
server_key_iv: "d236f5a8d8ef45c85854162adc83c0c3"
server_key_version: "encryptionkey@0"
I20260809 19:21:04.835227 13088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:04.835881 13108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:04.836040 13088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:04.836122 13088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "1725160d154144cc860d738c94013877"
format_stamp: "Formatted at 2026-08-09 19:21:04 on dist-test-slave-1vmg"
server_key: "e1517cc83e56d94dbecee5d8075be6df"
server_key_iv: "d236f5a8d8ef45c85854162adc83c0c3"
server_key_version: "encryptionkey@0"
I20260809 19:21:04.836179 13088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:04.843946 13088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:04.845569 13088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:04.845712 13088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:04.849792 13088 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:38515
I20260809 19:21:04.849839 13191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:38515 every 8 connection(s)
I20260809 19:21:04.850143 13088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:04.850894 13192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:04.853003 13192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877: Bootstrap starting.
I20260809 19:21:04.853628 13192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:04.853910 13192 log.cc:824] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:04.854554 13192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877: No bootstrap required, opened a new log
I20260809 19:21:04.854782 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 13088
I20260809 19:21:04.854875 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:04.855088 9581 external_mini_cluster.cc:1428] Setting key cb7b56e2147cf36794e4cff22d71ccf5
I20260809 19:21:04.856357 13192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1725160d154144cc860d738c94013877" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38515 } }
I20260809 19:21:04.856513 13192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:04.856588 13192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1725160d154144cc860d738c94013877, State: Initialized, Role: FOLLOWER
I20260809 19:21:04.856714 13192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1725160d154144cc860d738c94013877" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38515 } }
I20260809 19:21:04.856786 13192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:04.856807 13192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:04.856851 13192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:04.857791 13192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1725160d154144cc860d738c94013877" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38515 } }
I20260809 19:21:04.857896 13192 leader_election.cc:302] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1725160d154144cc860d738c94013877; no voters:
I20260809 19:21:04.858064 13192 leader_election.cc:288] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:04.858120 13198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:04.858282 13198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [term 1 LEADER]: Becoming Leader. State: Replica: 1725160d154144cc860d738c94013877, State: Running, Role: LEADER
I20260809 19:21:04.858309 13192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:04.858426 13198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1725160d154144cc860d738c94013877" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38515 } }
I20260809 19:21:04.858820 13199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1725160d154144cc860d738c94013877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1725160d154144cc860d738c94013877" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38515 } } }
I20260809 19:21:04.859125 13199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:04.859043 13201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1725160d154144cc860d738c94013877. Latest consensus state: current_term: 1 leader_uuid: "1725160d154144cc860d738c94013877" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1725160d154144cc860d738c94013877" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38515 } } }
I20260809 19:21:04.859428 13201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:04.859616 13214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:04.860307 13214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:21:04 dist-test-slave-1vmg krb5kdc[13067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:04.862370 13214 catalog_manager.cc:1347] Generated new cluster ID: 6208f93fa6884f8cb7de39735aff0c33
I20260809 19:21:04.862416 13214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:04.865758 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:04.855577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.9.91.126:38515)
0809 19:21:04.855758 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:04.855767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:04.855800 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:21:04.856320 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:04.856329 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:04.856402 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:04.856561 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:04.857259 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:04.858592 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:04.859625 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:04.860226 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:04.860337 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:04.862835 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:04.862839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:04.862839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:04.862849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:04.864360 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:04.864681 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:04.864684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:04.864685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:04.864782 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:04.864965 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:04.864966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:04.864966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:04.865197 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:04.865346 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:04.865438 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 19:21:04.865564 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260809 19:21:04.876489 13214 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:04.876940 13214 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:04.883090 13214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1725160d154144cc860d738c94013877: Generated new TSK 0
I20260809 19:21:04.883242 13214 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:21:04.917577 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38515
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:05.010072 13227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:05.010278 13227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:05.010301 13227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:05.011507 13227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:05.011548 13227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:05.011574 13227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:05.012991 13227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:05.013232 13227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:05.013438 13227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:05.018139 13235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:05.018123 13232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:05.018292 13227 server_base.cc:1034] running on GCE node
W20260809 19:21:05.018152 13233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:05.018517 13227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:05.018716 13227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:05.019866 13227 hybrid_clock.cc:648] HybridClock initialized: now 1786303265019854 us; error 31 us; skew 500 ppm
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:05.022285 13227 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:21:05.022656 13227 webserver.cc:460] Webserver started at http://127.9.91.65:37603/ using document root <none> and password file <none>
I20260809 19:21:05.022832 13227 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:05.022871 13227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:05.022998 13227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:05.024015 13227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "a9a756019c0f43aa9af5ae6238022a3a"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "992dcc62bdd7b89f33f0e462ff8a61da"
server_key_iv: "7eafc06ad5b20301ac0c45dc550a5614"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.024344 13227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "a9a756019c0f43aa9af5ae6238022a3a"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "992dcc62bdd7b89f33f0e462ff8a61da"
server_key_iv: "7eafc06ad5b20301ac0c45dc550a5614"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.025580 13227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:05.026180 13245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.026316 13227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:05.026398 13227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "a9a756019c0f43aa9af5ae6238022a3a"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "992dcc62bdd7b89f33f0e462ff8a61da"
server_key_iv: "7eafc06ad5b20301ac0c45dc550a5614"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.026458 13227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:05.039304 13227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:05.041348 13227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:05.041472 13227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:05.041662 13227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:05.041934 13227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:05.041968 13227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.041993 13227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:05.042017 13227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.047820 13227 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:45207
I20260809 19:21:05.047855 13421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:45207 every 8 connection(s)
I20260809 19:21:05.048126 13227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:05.052627 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 13227
I20260809 19:21:05.052734 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:05.052896 9581 external_mini_cluster.cc:1428] Setting key b307e64897fd92b519dace48d5a04bf0
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:05.056965 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:05.048746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:49565 (local address 127.9.91.126:38515)
0809 19:21:05.048810 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:05.048812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:05.048981 (+ 169us) server_negotiation.cc:407] Connection header received
0809 19:21:05.049448 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:05.049449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:05.049462 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:05.049513 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:05.050402 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.050938 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.051871 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.052094 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.052150 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:05.054200 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:05.054201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:05.054201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:05.054209 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:05.055696 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:05.056059 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:05.056060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:05.056060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:05.056126 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:05.056336 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:05.056337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:05.056337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:05.056455 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:05.056513 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:05.056776 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 19:21:05.056872 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:21:05.057101 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:05.048707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38515 (local address 127.9.91.65:49565)
0809 19:21:05.048866 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:05.048910 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:05.049355 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:05.049577 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:05.049581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:05.050116 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:05.050311 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:05.050316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.051016 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.051018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:05.051788 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:05.051790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.052165 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.052166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:05.052287 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:05.052711 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:05.052719 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:05.054112 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:05.055782 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:05.055789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:05.055791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:05.055992 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:05.056185 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:05.056185 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:05.056186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:05.056281 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:05.056525 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:05.056533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:05.056659 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 19:21:05.056747 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:21:05.056890 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260809 19:21:05.057523 13422 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38515
I20260809 19:21:05.057612 13422 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:05.057772 13422 heartbeater.cc:511] Master 127.9.91.126:38515 requested a full tablet report, sending...
I20260809 19:21:05.058301 13133 ts_manager.cc:194] Registered new tserver with Master: a9a756019c0f43aa9af5ae6238022a3a (127.9.91.65:45207)
I20260809 19:21:05.059231 13133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:49565
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:21:05.075160 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38515
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:05.152804 13429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:05.152985 13429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:05.153015 13429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:05.154116 13429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:05.154155 13429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:05.154188 13429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:05.155567 13429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:05.155781 13429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:05.155974 13429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:05.160602 13437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:05.160593 13436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:05.160593 13440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:05.160957 13429 server_base.cc:1034] running on GCE node
I20260809 19:21:05.161191 13429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:05.161439 13429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:05.162588 13429 hybrid_clock.cc:648] HybridClock initialized: now 1786303265162574 us; error 38 us; skew 500 ppm
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:05.168751 13429 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:21:05.169164 13429 webserver.cc:460] Webserver started at http://127.9.91.66:39463/ using document root <none> and password file <none>
I20260809 19:21:05.169354 13429 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:05.169401 13429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:05.169521 13429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:05.170451 13429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "59fbe67e315c4f829b483f78764e0b73"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "2a96d4dc8eee58988947003778679c6e"
server_key_iv: "26aac2c6e22f9c5e32f1f8103c3ad071"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.170841 13429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "59fbe67e315c4f829b483f78764e0b73"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "2a96d4dc8eee58988947003778679c6e"
server_key_iv: "26aac2c6e22f9c5e32f1f8103c3ad071"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.172204 13429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:05.172827 13449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.172967 13429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:05.173045 13429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "59fbe67e315c4f829b483f78764e0b73"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "2a96d4dc8eee58988947003778679c6e"
server_key_iv: "26aac2c6e22f9c5e32f1f8103c3ad071"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.173107 13429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:05.186491 13429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:05.188102 13429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:05.188222 13429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:05.188400 13429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:05.188668 13429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:05.188707 13429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.188732 13429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:05.188757 13429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.194396 13429 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:46625
I20260809 19:21:05.194427 13619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:46625 every 8 connection(s)
I20260809 19:21:05.194695 13429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:05.199049 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 13429
I20260809 19:21:05.199149 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:05.199349 9581 external_mini_cluster.cc:1428] Setting key 00bcfef6a4c472b2a36d2a1d524db644
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:05.204344 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:05.195332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:52365 (local address 127.9.91.126:38515)
0809 19:21:05.195387 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:05.195389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:05.195570 (+ 181us) server_negotiation.cc:407] Connection header received
0809 19:21:05.195979 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:05.195980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:05.195989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:05.196039 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:05.197140 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.197910 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.199090 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.199364 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.199401 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:05.201667 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:05.201668 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:05.201669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:05.201678 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:05.203056 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:05.203422 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:05.203423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:05.203423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:05.203482 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:05.203696 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:05.203697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:05.203697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:05.203803 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:05.203860 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:05.204158 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 19:21:05.204248 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:21:05.204452 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:05.195303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38515 (local address 127.9.91.66:52365)
0809 19:21:05.195460 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:05.195498 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:05.195911 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:05.196095 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:05.196097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:05.196762 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:05.197012 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:05.197018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.198026 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.198027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:05.198986 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:05.198990 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.199524 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.199525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:05.199669 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:05.200101 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:05.200109 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:05.201580 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:05.203134 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:05.203141 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:05.203143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:05.203347 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:05.203540 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:05.203541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:05.203541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:05.203639 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:05.203858 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:05.203866 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:05.203992 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 19:21:05.204087 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 19:21:05.204249 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260809 19:21:05.204901 13620 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38515
I20260809 19:21:05.204990 13620 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:05.205171 13620 heartbeater.cc:511] Master 127.9.91.126:38515 requested a full tablet report, sending...
I20260809 19:21:05.205549 13133 ts_manager.cc:194] Registered new tserver with Master: 59fbe67e315c4f829b483f78764e0b73 (127.9.91.66:46625)
I20260809 19:21:05.206027 13133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:52365
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:21:05.223397 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38515
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:05.301761 13627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:05.301935 13627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:05.301957 13627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:05.303092 13627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:05.303129 13627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:05.303160 13627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:05.304544 13627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:05.304741 13627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:05.304932 13627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:05.309563 13642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:05.309544 13638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:05.309535 13635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:05.309682 13627 server_base.cc:1034] running on GCE node
I20260809 19:21:05.309937 13627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:05.310161 13627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:05.311298 13627 hybrid_clock.cc:648] HybridClock initialized: now 1786303265311285 us; error 25 us; skew 500 ppm
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:05.313602 13627 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:21:05.313972 13627 webserver.cc:460] Webserver started at http://127.9.91.67:44439/ using document root <none> and password file <none>
I20260809 19:21:05.314162 13627 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:05.314208 13627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:05.314323 13627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:05.315357 13627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "2b592f4bca25487986818b15d23bc249"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "0169047e711ffb29e2535dbb3a62aca5"
server_key_iv: "11bcedf5214f11cfb848e4e643f674e0"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.315783 13627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "2b592f4bca25487986818b15d23bc249"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "0169047e711ffb29e2535dbb3a62aca5"
server_key_iv: "11bcedf5214f11cfb848e4e643f674e0"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.317052 13627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:05.317670 13651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.317816 13627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:05.317893 13627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "2b592f4bca25487986818b15d23bc249"
format_stamp: "Formatted at 2026-08-09 19:21:05 on dist-test-slave-1vmg"
server_key: "0169047e711ffb29e2535dbb3a62aca5"
server_key_iv: "11bcedf5214f11cfb848e4e643f674e0"
server_key_version: "encryptionkey@0"
I20260809 19:21:05.317950 13627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:05.348311 13627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:05.350132 13627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:05.350255 13627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:05.350467 13627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:05.350746 13627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:05.350780 13627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.350806 13627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:05.350831 13627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:05.356827 13627 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:39683
I20260809 19:21:05.356885 13807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:39683 every 8 connection(s)
I20260809 19:21:05.357152 13627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:05.357484 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 13627
I20260809 19:21:05.357590 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:05.357785 9581 external_mini_cluster.cc:1428] Setting key 2b432e545b35d103c87977911048868f
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303265, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:05.365497 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:05.357862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:41461 (local address 127.9.91.126:38515)
0809 19:21:05.357912 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:05.357914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:05.358135 (+ 221us) server_negotiation.cc:407] Connection header received
0809 19:21:05.358703 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:05.358704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:05.358715 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:05.358754 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:05.359727 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.360348 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.361261 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.361440 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.361483 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:05.363357 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:05.363358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:05.363359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:05.363365 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:05.364553 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:05.364852 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:05.364853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:05.364853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:05.364900 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:05.365066 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:05.365067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:05.365067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:05.365157 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:05.365200 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:05.365383 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 19:21:05.365435 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:21:05.365674 13812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:05.357848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38515 (local address 127.9.91.67:41461)
0809 19:21:05.358001 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:05.358046 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:05.358614 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:05.358815 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:05.358819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:05.359391 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:05.359651 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:05.359658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.360424 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.360425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:05.361203 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:05.361205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:05.361490 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:05.361490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:05.361587 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:05.361996 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:05.362004 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:05.363281 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:05.364610 (+ 1329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:05.364612 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:05.364614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:05.364802 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:05.364942 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:05.364943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:05.364943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:05.365026 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:05.365198 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:05.365200 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:05.365302 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 19:21:05.365371 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:21:05.365479 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260809 19:21:05.366102 13808 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38515
I20260809 19:21:05.366185 13808 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:05.366344 13808 heartbeater.cc:511] Master 127.9.91.126:38515 requested a full tablet report, sending...
I20260809 19:21:05.366678 13133 ts_manager.cc:194] Registered new tserver with Master: 2b592f4bca25487986818b15d23bc249 (127.9.91.67:39683)
I20260809 19:21:05.367056 13133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:41461
I20260809 19:21:05.369212 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:21:05.381376 13196 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:05.380631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.9.91.126:38515)
0809 19:21:05.380687 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:05.380689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:05.380764 (+ 75us) server_negotiation.cc:407] Connection header received
0809 19:21:05.380826 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:05.380826 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:05.380836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:05.380880 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:05.381334 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55556: BlockingRecv error: recv got EOF from 127.0.0.1:55556 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:21:05.382330 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 13227
I20260809 19:21:05.409623 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 13429
I20260809 19:21:05.420567 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 13627
I20260809 19:21:05.426841 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 13088
2026-08-09T19:21:05Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3398 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13826](info): set up 2 sockets
Aug 09 19:21:05 dist-test-slave-1vmg krb5kdc[13826](info): commencing operation
krb5kdc: starting...
W20260809 19:21:06.552176 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:06 dist-test-slave-1vmg krb5kdc[13826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:06.663007 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46703
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:46703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:06.741386 13845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:06.741564 13845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:06.741592 13845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:06.742710 13845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:06.742748 13845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:06.742767 13845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:06.742781 13845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:06.742797 13845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:06.744320 13845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:46703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:06.744542 13845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:06.744745 13845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:06.749327 13856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:06.749306 13854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:06.749325 13853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:06.749531 13845 server_base.cc:1034] running on GCE node
I20260809 19:21:06.749778 13845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:06.750056 13845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:06.751211 13845 hybrid_clock.cc:648] HybridClock initialized: now 1786303266751191 us; error 32 us; skew 500 ppm
Aug 09 19:21:06 dist-test-slave-1vmg krb5kdc[13826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303266, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:06.753598 13845 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:06.754007 13845 webserver.cc:460] Webserver started at http://127.9.91.126:43105/ using document root <none> and password file <none>
I20260809 19:21:06.754200 13845 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:06.754251 13845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:06.754411 13845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:06.755604 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "b43ea773f749487195ba40d406c841e5"
format_stamp: "Formatted at 2026-08-09 19:21:06 on dist-test-slave-1vmg"
server_key: "4b44f7111f17d8880cdb1f04e459b192"
server_key_iv: "1c202b9f489222ae4732caa2e1957286"
server_key_version: "encryptionkey@0"
I20260809 19:21:06.755945 13845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "b43ea773f749487195ba40d406c841e5"
format_stamp: "Formatted at 2026-08-09 19:21:06 on dist-test-slave-1vmg"
server_key: "4b44f7111f17d8880cdb1f04e459b192"
server_key_iv: "1c202b9f489222ae4732caa2e1957286"
server_key_version: "encryptionkey@0"
I20260809 19:21:06.757354 13845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:21:06.758116 13867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:06.758279 13845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:06.758369 13845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "b43ea773f749487195ba40d406c841e5"
format_stamp: "Formatted at 2026-08-09 19:21:06 on dist-test-slave-1vmg"
server_key: "4b44f7111f17d8880cdb1f04e459b192"
server_key_iv: "1c202b9f489222ae4732caa2e1957286"
server_key_version: "encryptionkey@0"
I20260809 19:21:06.758440 13845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:06.770529 13845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:06.772184 13845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:06.772321 13845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:06.776453 13845 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:46703
I20260809 19:21:06.776484 13950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:46703 every 8 connection(s)
I20260809 19:21:06.776826 13845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:06.777642 13951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:06.779769 13951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5: Bootstrap starting.
I20260809 19:21:06.780309 13951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:06.780540 13951 log.cc:824] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:06.781239 13951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5: No bootstrap required, opened a new log
I20260809 19:21:06.782730 13951 raft_consensus.cc:348] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43ea773f749487195ba40d406c841e5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46703 } }
I20260809 19:21:06.782851 13951 raft_consensus.cc:374] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:06.782871 13951 raft_consensus.cc:729] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b43ea773f749487195ba40d406c841e5, State: Initialized, Role: FOLLOWER
I20260809 19:21:06.783005 13951 consensus_queue.cc:260] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43ea773f749487195ba40d406c841e5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46703 } }
I20260809 19:21:06.783064 13951 raft_consensus.cc:388] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:06.783084 13951 raft_consensus.cc:482] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:06.783119 13951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:06.783807 13951 raft_consensus.cc:504] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43ea773f749487195ba40d406c841e5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46703 } }
I20260809 19:21:06.783891 13951 leader_election.cc:302] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b43ea773f749487195ba40d406c841e5; no voters:
I20260809 19:21:06.784008 13951 leader_election.cc:288] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:06.784060 13957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:06.784225 13951 sys_catalog.cc:564] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:06.784235 13957 raft_consensus.cc:686] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [term 1 LEADER]: Becoming Leader. State: Replica: b43ea773f749487195ba40d406c841e5, State: Running, Role: LEADER
I20260809 19:21:06.784353 13957 consensus_queue.cc:237] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43ea773f749487195ba40d406c841e5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46703 } }
I20260809 19:21:06.784695 13963 sys_catalog.cc:455] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b43ea773f749487195ba40d406c841e5. Latest consensus state: current_term: 1 leader_uuid: "b43ea773f749487195ba40d406c841e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43ea773f749487195ba40d406c841e5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46703 } } }
I20260809 19:21:06.784864 13960 sys_catalog.cc:455] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b43ea773f749487195ba40d406c841e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b43ea773f749487195ba40d406c841e5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46703 } } }
I20260809 19:21:06.785022 13960 sys_catalog.cc:458] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:06.784934 13963 sys_catalog.cc:458] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:06.785267 13978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:06.785714 13978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:06.786980 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 13845
I20260809 19:21:06.787070 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:06.787253 9581 external_mini_cluster.cc:1428] Setting key 616edd3b353df2a226f1352ece739bb8
I20260809 19:21:06.787248 13978 catalog_manager.cc:1347] Generated new cluster ID: 71be79853f0e4035b4ba113e77bf8716
I20260809 19:21:06.787293 13978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:21:06 dist-test-slave-1vmg krb5kdc[13826](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:06.805383 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:06.787665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.9.91.126:46703)
0809 19:21:06.795034 (+ 7369us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:06.795043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:06.795073 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:21:06.795175 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:06.795180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:06.795238 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:06.795359 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:06.796119 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:06.797429 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:06.798482 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:06.799108 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:06.799199 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:06.801869 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:06.801873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:06.801873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:06.801886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:06.803724 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:06.804077 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:06.804080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:06.804080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:06.804195 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:06.804363 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:06.804364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:06.804364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:06.804671 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:06.804803 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:06.804893 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 19:21:06.805023 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7314,"thread_start_us":67,"threads_started":1}
I20260809 19:21:06.806216 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:21:06.810233 13978 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:06.810679 13978 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:06.819483 13978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b43ea773f749487195ba40d406c841e5: Generated new TSK 0
I20260809 19:21:06.819682 13978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:06.827044 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:06.823328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60312 (local address 127.9.91.126:46703)
0809 19:21:06.823368 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:06.823370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:06.823459 (+ 89us) server_negotiation.cc:407] Connection header received
0809 19:21:06.823531 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:06.823532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:06.823549 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:06.823595 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:06.823992 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:06.824752 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:06.825764 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:06.825962 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:06.826005 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:06.826613 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:06.826614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:06.826614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:06.826791 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:06.826806 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:06.826812 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:06.826857 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:06.826974 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 19:21:06.826997 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:21:06.829129 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 13845
2026-08-09T19:21:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1406 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:06 dist-test-slave-1vmg krb5kdc[14008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:06 dist-test-slave-1vmg krb5kdc[14008](info): set up 2 sockets
Aug 09 19:21:06 dist-test-slave-1vmg krb5kdc[14008](info): commencing operation
krb5kdc: starting...
W20260809 19:21:07.949108 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:08.055413 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:42235
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:42235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:08.133597 14026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:08.133790 14026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:08.133821 14026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:08.134945 14026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:08.134984 14026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:08.135001 14026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:08.135017 14026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:08.135032 14026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:08.136469 14026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:42235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:42235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:08.136710 14026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:08.136909 14026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:08.141366 14032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:08.141450 14026 server_base.cc:1034] running on GCE node
W20260809 19:21:08.141467 14033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.141412 14037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:08.141744 14026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:08.141994 14026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:08.143122 14026 hybrid_clock.cc:648] HybridClock initialized: now 1786303268143109 us; error 31 us; skew 500 ppm
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:08.145332 14026 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:08.145740 14026 webserver.cc:460] Webserver started at http://127.9.91.126:45527/ using document root <none> and password file <none>
I20260809 19:21:08.145929 14026 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:08.145977 14026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:08.146098 14026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:08.147153 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "6f7b593998eae01fad9b289822138101"
server_key_iv: "2844cf97a4660bd49b8cd2acf638a4aa"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.147476 14026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "6f7b593998eae01fad9b289822138101"
server_key_iv: "2844cf97a4660bd49b8cd2acf638a4aa"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.148751 14026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:08.149375 14045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.149519 14026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:08.149600 14026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "6f7b593998eae01fad9b289822138101"
server_key_iv: "2844cf97a4660bd49b8cd2acf638a4aa"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.149662 14026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:08.167797 14026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:08.169332 14026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:08.169459 14026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:08.173177 14026 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:42235
I20260809 19:21:08.173213 14112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:42235 every 8 connection(s)
I20260809 19:21:08.173496 14026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:08.174407 14116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:08.176437 14116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c: Bootstrap starting.
I20260809 19:21:08.176976 14116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:08.177215 14116 log.cc:824] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:08.177805 14116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c: No bootstrap required, opened a new log
I20260809 19:21:08.179168 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 14026
I20260809 19:21:08.179262 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:08.179416 9581 external_mini_cluster.cc:1428] Setting key 45517313b2c0ca3587b102b20839ab2b
I20260809 19:21:08.179355 14116 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42235 } }
I20260809 19:21:08.179469 14116 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:08.179512 14116 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1e845c5c8ba43b5b7cd98ac2d8bc41c, State: Initialized, Role: FOLLOWER
I20260809 19:21:08.179646 14116 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42235 } }
I20260809 19:21:08.179731 14116 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:08.179775 14116 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:08.179813 14116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:08.180766 14116 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42235 } }
I20260809 19:21:08.180852 14116 leader_election.cc:302] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1e845c5c8ba43b5b7cd98ac2d8bc41c; no voters:
I20260809 19:21:08.181023 14116 leader_election.cc:288] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:08.181066 14124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:08.181164 14124 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [term 1 LEADER]: Becoming Leader. State: Replica: e1e845c5c8ba43b5b7cd98ac2d8bc41c, State: Running, Role: LEADER
I20260809 19:21:08.181279 14116 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:08.181308 14124 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42235 } }
I20260809 19:21:08.181618 14124 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1e845c5c8ba43b5b7cd98ac2d8bc41c. Latest consensus state: current_term: 1 leader_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42235 } } }
I20260809 19:21:08.181843 14124 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:08.181764 14125 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1e845c5c8ba43b5b7cd98ac2d8bc41c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42235 } } }
I20260809 19:21:08.182240 14125 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:08.182379 14137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:08.182881 14137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:08.184477 14137 catalog_manager.cc:1347] Generated new cluster ID: 4dd05303ce514d1fb29ad17be4bc312b
I20260809 19:21:08.184528 14137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:08.189963 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.179815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.9.91.126:42235)
0809 19:21:08.179994 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:08.180003 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:08.180046 (+ 43us) server_negotiation.cc:407] Connection header received
0809 19:21:08.180165 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:08.180173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:08.180254 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:08.180400 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:08.180965 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.182195 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.183298 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.183760 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.183861 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.186505 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:08.186509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:08.186510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:08.186521 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:08.188391 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.188765 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.188768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.188769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.188885 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.189056 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.189057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.189057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.189414 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:08.189574 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:08.189639 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:21:08.189775 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260809 19:21:08.201376 14137 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:08.201871 14137 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:08.206216 14137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1e845c5c8ba43b5b7cd98ac2d8bc41c: Generated new TSK 0
I20260809 19:21:08.206364 14137 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:21:08.235212 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42235
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:08.317740 14155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:08.317930 14155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:08.317951 14155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:08.319104 14155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:08.319144 14155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:08.319171 14155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:08.320598 14155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:08.320807 14155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:08.321012 14155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:08.325629 14166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.325630 14163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:08.325843 14155 server_base.cc:1034] running on GCE node
W20260809 19:21:08.326025 14162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:08.326227 14155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:08.326476 14155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:08.327605 14155 hybrid_clock.cc:648] HybridClock initialized: now 1786303268327591 us; error 30 us; skew 500 ppm
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:08.329890 14155 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:21:08.330269 14155 webserver.cc:460] Webserver started at http://127.9.91.65:37469/ using document root <none> and password file <none>
I20260809 19:21:08.330456 14155 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:08.330502 14155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:08.330617 14155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:08.331625 14155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "b9b643ab3ef44975ba3bc08f6ff71ef4"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "d5d079307dcfa86501db5c487066ef59"
server_key_iv: "cb088b8295c0bb8d8c3960597efec679"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.331951 14155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "b9b643ab3ef44975ba3bc08f6ff71ef4"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "d5d079307dcfa86501db5c487066ef59"
server_key_iv: "cb088b8295c0bb8d8c3960597efec679"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.333110 14155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:21:08.333714 14175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.333860 14155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:08.333941 14155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "b9b643ab3ef44975ba3bc08f6ff71ef4"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "d5d079307dcfa86501db5c487066ef59"
server_key_iv: "cb088b8295c0bb8d8c3960597efec679"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.333997 14155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:08.348176 14155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:08.349917 14155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:08.350036 14155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:08.350225 14155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:08.350478 14155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:08.350509 14155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.350528 14155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:08.350551 14155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.356170 14155 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:34951
I20260809 19:21:08.356245 14346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:34951 every 8 connection(s)
I20260809 19:21:08.356487 14155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:08.359193 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 14155
I20260809 19:21:08.359299 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:08.359508 9581 external_mini_cluster.cc:1428] Setting key fffa531a57e5824f2bf176625a4cc573
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:08.365605 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.357129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35075 (local address 127.9.91.126:42235)
0809 19:21:08.357194 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:08.357196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:08.357379 (+ 183us) server_negotiation.cc:407] Connection header received
0809 19:21:08.357779 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:08.357779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:08.357792 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:08.357839 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:08.358629 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.359162 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.360485 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.360676 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.360730 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.363048 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:08.363049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:08.363049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:08.363056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:08.364339 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.364696 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.364698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.364698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.364772 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.365018 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.365020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.365020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.365156 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:08.365219 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:08.365432 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 19:21:08.365517 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:21:08.365639 14352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.357096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:42235 (local address 127.9.91.65:35075)
0809 19:21:08.357266 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:08.357304 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:08.357708 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:08.357885 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:08.357888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:08.358398 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:08.358564 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:08.358568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.359251 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.359252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:08.360398 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:08.360401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.360754 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.360755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:08.360881 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.361294 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:08.361304 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:08.362967 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:08.364411 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:08.364417 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:08.364419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:08.364623 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:08.364837 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:08.364838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:08.364838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:08.364959 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:08.365217 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:08.365220 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:08.365321 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 19:21:08.365393 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 19:21:08.365501 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260809 19:21:08.366124 14348 heartbeater.cc:347] Connected to a master server at 127.9.91.126:42235
I20260809 19:21:08.366219 14348 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:08.366375 14348 heartbeater.cc:511] Master 127.9.91.126:42235 requested a full tablet report, sending...
I20260809 19:21:08.366894 14067 ts_manager.cc:194] Registered new tserver with Master: b9b643ab3ef44975ba3bc08f6ff71ef4 (127.9.91.65:34951)
I20260809 19:21:08.367781 14067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:35075
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:21:08.384966 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42235
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:08.462730 14356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:08.462908 14356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:08.462953 14356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:08.464068 14356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:08.464107 14356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:08.464140 14356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:08.465509 14356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:08.465741 14356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:08.465929 14356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:08.470486 14362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.470528 14364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.470485 14366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:08.470574 14356 server_base.cc:1034] running on GCE node
I20260809 19:21:08.470855 14356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:08.471076 14356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:08.472213 14356 hybrid_clock.cc:648] HybridClock initialized: now 1786303268472194 us; error 29 us; skew 500 ppm
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:08.474400 14356 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:21:08.474790 14356 webserver.cc:460] Webserver started at http://127.9.91.66:36751/ using document root <none> and password file <none>
I20260809 19:21:08.475006 14356 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:08.475055 14356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:08.475175 14356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:08.476099 14356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "8ef16d52cb2446b3b0705b53cd87b6c0"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "1005451f89fa0b2a8027836e38295fc2"
server_key_iv: "cea8e24a1d918fa499dd3d2fd83dad16"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.476430 14356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "8ef16d52cb2446b3b0705b53cd87b6c0"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "1005451f89fa0b2a8027836e38295fc2"
server_key_iv: "cea8e24a1d918fa499dd3d2fd83dad16"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.477648 14356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:08.478230 14377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.478375 14356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:08.478459 14356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "8ef16d52cb2446b3b0705b53cd87b6c0"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "1005451f89fa0b2a8027836e38295fc2"
server_key_iv: "cea8e24a1d918fa499dd3d2fd83dad16"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.478509 14356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:08.514654 14356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:08.516290 14356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:08.516392 14356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:08.516564 14356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:08.516810 14356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:08.516841 14356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.516860 14356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:08.516876 14356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.522516 14356 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:38919
I20260809 19:21:08.522583 14549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:38919 every 8 connection(s)
I20260809 19:21:08.522847 14356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:08.528968 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 14356
I20260809 19:21:08.529074 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:08.529289 9581 external_mini_cluster.cc:1428] Setting key 3a2f6f35a3d02100aa0da944120375e8
I20260809 19:21:08.531512 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.523515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:51571 (local address 127.9.91.126:42235)
0809 19:21:08.523559 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:08.523560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:08.523747 (+ 187us) server_negotiation.cc:407] Connection header received
0809 19:21:08.524236 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:08.524237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:08.524249 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:08.524297 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:08.525181 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.525672 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.526615 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.526812 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.526867 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.528802 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:08.528804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:08.528805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:08.528814 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:08.530271 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.530662 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.530663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.530663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.530728 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.530942 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.530942 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.530943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.531071 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:08.531136 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:08.531344 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:21:08.531426 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:21:08.531570 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.523479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:42235 (local address 127.9.91.66:51571)
0809 19:21:08.523637 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:08.523669 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:08.524139 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:08.524375 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:08.524379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:08.524912 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:08.525088 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:08.525092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.525755 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.525757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:08.526532 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:08.526535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.526883 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.526884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:08.527025 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.527414 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:08.527421 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:08.528721 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:08.530345 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:08.530351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:08.530353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:08.530594 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:08.530784 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:08.530785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:08.530785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:08.530875 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:08.531125 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:08.531129 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:08.531234 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 19:21:08.531312 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:21:08.531422 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260809 19:21:08.532194 14550 heartbeater.cc:347] Connected to a master server at 127.9.91.126:42235
I20260809 19:21:08.532279 14550 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:08.532435 14550 heartbeater.cc:511] Master 127.9.91.126:42235 requested a full tablet report, sending...
I20260809 19:21:08.532775 14067 ts_manager.cc:194] Registered new tserver with Master: 8ef16d52cb2446b3b0705b53cd87b6c0 (127.9.91.66:38919)
I20260809 19:21:08.533110 14067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:51571
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:21:08.552774 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42235
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:36769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:08.630553 14563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:08.630725 14563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:08.630746 14563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:08.631834 14563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:08.631866 14563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:08.631891 14563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:08.633222 14563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:08.633388 14563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:08.633548 14563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:08.638007 14568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.638059 14572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:08.638056 14569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:08.638177 14563 server_base.cc:1034] running on GCE node
I20260809 19:21:08.638370 14563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:08.638677 14563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:08.639832 14563 hybrid_clock.cc:648] HybridClock initialized: now 1786303268639819 us; error 25 us; skew 500 ppm
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:08.642078 14563 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:21:08.642555 14563 webserver.cc:460] Webserver started at http://127.9.91.67:45523/ using document root <none> and password file <none>
I20260809 19:21:08.642740 14563 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:08.642778 14563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:08.642863 14563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:08.643790 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "7edf459c4c7847968dd116bdab6f1964"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "bb9252a189071905a16e5279e7905f59"
server_key_iv: "fd7877ef033b0597cab088f9aedc3ff4"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.644064 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "7edf459c4c7847968dd116bdab6f1964"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "bb9252a189071905a16e5279e7905f59"
server_key_iv: "fd7877ef033b0597cab088f9aedc3ff4"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.645470 14563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:08.646075 14579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.646224 14563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:08.646302 14563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "7edf459c4c7847968dd116bdab6f1964"
format_stamp: "Formatted at 2026-08-09 19:21:08 on dist-test-slave-1vmg"
server_key: "bb9252a189071905a16e5279e7905f59"
server_key_iv: "fd7877ef033b0597cab088f9aedc3ff4"
server_key_version: "encryptionkey@0"
I20260809 19:21:08.646358 14563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:08.653518 14563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:08.655431 14563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:08.655551 14563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:08.655735 14563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:08.656008 14563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:08.656046 14563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.656072 14563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:08.656097 14563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:08.661609 14563 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:45973
I20260809 19:21:08.661669 14741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:45973 every 8 connection(s)
I20260809 19:21:08.661921 14563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:08.666745 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 14563
I20260809 19:21:08.666847 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:08.667079 9581 external_mini_cluster.cc:1428] Setting key 91b8788ba32d332f8b447853cdba7573
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:08.670082 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.662522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:48993 (local address 127.9.91.126:42235)
0809 19:21:08.662569 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:08.662571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:08.662782 (+ 211us) server_negotiation.cc:407] Connection header received
0809 19:21:08.663267 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:08.663268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:08.663280 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:08.663331 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:08.664112 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.664591 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.665461 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.665634 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.665677 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.667751 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:08.667752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:08.667752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:08.667760 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:08.668769 (+ 1009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.669220 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.669221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.669222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.669276 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.669496 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.669497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.669497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.669609 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:08.669665 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:08.669915 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 19:21:08.669995 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:21:08.670337 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.662498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:42235 (local address 127.9.91.67:48993)
0809 19:21:08.662677 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:08.662710 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:08.663165 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:08.663378 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:08.663381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:08.663880 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:08.664050 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:08.664054 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.664646 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.664647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:08.665400 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:08.665403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.665686 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.665686 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:08.665772 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.666140 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:08.666147 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:08.667666 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:08.668848 (+ 1182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:08.668855 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:08.668857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:08.669148 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:08.669331 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:08.669332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:08.669333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:08.669440 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:08.669666 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:08.669673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:08.669799 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 19:21:08.669882 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:21:08.670033 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260809 19:21:08.670696 14743 heartbeater.cc:347] Connected to a master server at 127.9.91.126:42235
I20260809 19:21:08.670773 14743 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:08.670902 14743 heartbeater.cc:511] Master 127.9.91.126:42235 requested a full tablet report, sending...
I20260809 19:21:08.671247 14067 ts_manager.cc:194] Registered new tserver with Master: 7edf459c4c7847968dd116bdab6f1964 (127.9.91.67:45973)
I20260809 19:21:08.671584 14067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:48993
I20260809 19:21:08.678758 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303268, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:08.701558 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:08.695294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.9.91.126:42235)
0809 19:21:08.695354 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:08.695356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:08.695433 (+ 77us) server_negotiation.cc:407] Connection header received
0809 19:21:08.695508 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:08.695509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:08.695519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:08.695577 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:08.696112 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.696578 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.697322 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:08.697511 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:08.697572 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:08.699568 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:08.699569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:08.699570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:08.699576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:08.700573 (+ 997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.700866 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.700867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.700867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.700917 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:08.701078 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:08.701079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:08.701079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:08.701212 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:08.701298 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:08.701403 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 19:21:08.701474 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 19:21:08.701748 14067 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42340
I20260809 19:21:08.702134 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 14155
I20260809 19:21:08.719754 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 14356
I20260809 19:21:08.729146 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 14563
I20260809 19:21:08.742179 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 14026
2026-08-09T19:21:08Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1908 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14758](info): set up 2 sockets
Aug 09 19:21:08 dist-test-slave-1vmg krb5kdc[14758](info): commencing operation
krb5kdc: starting...
W20260809 19:21:09.858357 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:09 dist-test-slave-1vmg krb5kdc[14758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:09.965991 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:39353
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:39353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:10.043332 14776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:10.043511 14776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:10.043541 14776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:10.044659 14776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:10.044708 14776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:10.044727 14776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:10.044744 14776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:10.044759 14776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:10.046226 14776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:39353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:39353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:10.046446 14776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:10.046639 14776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:10.051122 14782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:10.051200 14776 server_base.cc:1034] running on GCE node
W20260809 19:21:10.051144 14781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:10.051156 14784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:10.051529 14776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:10.051783 14776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:10.052915 14776 hybrid_clock.cc:648] HybridClock initialized: now 1786303270052904 us; error 27 us; skew 500 ppm
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:10.055238 14776 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:10.055619 14776 webserver.cc:460] Webserver started at http://127.9.91.126:44415/ using document root <none> and password file <none>
I20260809 19:21:10.055781 14776 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:10.055817 14776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:10.055902 14776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:10.056792 14776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "baf41b33f6f041ee934e2d2a4460a309"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "8683e3eb73eaa26a51d18425ed84324e"
server_key_iv: "5bdd328c0a3ba688803caa39d9f3c1f5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.057066 14776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "baf41b33f6f041ee934e2d2a4460a309"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "8683e3eb73eaa26a51d18425ed84324e"
server_key_iv: "5bdd328c0a3ba688803caa39d9f3c1f5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.058171 14776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:21:10.058772 14792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.058938 14776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.059015 14776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "baf41b33f6f041ee934e2d2a4460a309"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "8683e3eb73eaa26a51d18425ed84324e"
server_key_iv: "5bdd328c0a3ba688803caa39d9f3c1f5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.059065 14776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:10.075182 14776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:10.076656 14776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:10.076762 14776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:10.080461 14776 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:39353
I20260809 19:21:10.080490 14877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:39353 every 8 connection(s)
I20260809 19:21:10.080945 14776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:10.081532 14878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:10.083549 14878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309: Bootstrap starting.
I20260809 19:21:10.084067 14878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:10.084306 14878 log.cc:824] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:10.084899 14878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309: No bootstrap required, opened a new log
I20260809 19:21:10.086356 14878 raft_consensus.cc:348] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf41b33f6f041ee934e2d2a4460a309" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39353 } }
I20260809 19:21:10.086475 14878 raft_consensus.cc:374] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:10.086503 14878 raft_consensus.cc:729] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf41b33f6f041ee934e2d2a4460a309, State: Initialized, Role: FOLLOWER
I20260809 19:21:10.086597 14878 consensus_queue.cc:260] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf41b33f6f041ee934e2d2a4460a309" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39353 } }
I20260809 19:21:10.086645 14878 raft_consensus.cc:388] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:10.086669 14878 raft_consensus.cc:482] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:10.086696 14878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:10.087507 14878 raft_consensus.cc:504] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf41b33f6f041ee934e2d2a4460a309" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39353 } }
I20260809 19:21:10.087591 14878 leader_election.cc:302] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: baf41b33f6f041ee934e2d2a4460a309; no voters:
I20260809 19:21:10.087713 14878 leader_election.cc:288] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:10.087765 14881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:10.087914 14878 sys_catalog.cc:564] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:10.087921 14881 raft_consensus.cc:686] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [term 1 LEADER]: Becoming Leader. State: Replica: baf41b33f6f041ee934e2d2a4460a309, State: Running, Role: LEADER
I20260809 19:21:10.088090 14881 consensus_queue.cc:237] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf41b33f6f041ee934e2d2a4460a309" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39353 } }
I20260809 19:21:10.088418 14886 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [sys.catalog]: SysCatalogTable state changed. Reason: New leader baf41b33f6f041ee934e2d2a4460a309. Latest consensus state: current_term: 1 leader_uuid: "baf41b33f6f041ee934e2d2a4460a309" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf41b33f6f041ee934e2d2a4460a309" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39353 } } }
I20260809 19:21:10.088492 14886 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:10.088476 14884 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "baf41b33f6f041ee934e2d2a4460a309" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf41b33f6f041ee934e2d2a4460a309" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39353 } } }
I20260809 19:21:10.088676 14884 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:10.088881 14896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:10.089433 14896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:10.089819 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 14776
I20260809 19:21:10.089910 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:10.090076 9581 external_mini_cluster.cc:1428] Setting key aca9c9c159c088407bfbae0fc7ae1864
I20260809 19:21:10.091799 14896 catalog_manager.cc:1347] Generated new cluster ID: 59115dcd61444536981e9e9b8d3f1947
I20260809 19:21:10.091848 14896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:10.096563 14896 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:10.097007 14896 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:10.100268 14896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P baf41b33f6f041ee934e2d2a4460a309: Generated new TSK 0
I20260809 19:21:10.100415 14896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:10.101087 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.090459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43602 (local address 127.9.91.126:39353)
0809 19:21:10.090599 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:10.090607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:10.090638 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:21:10.090795 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:10.090802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:10.090856 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:10.091009 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:10.091714 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.093128 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.094212 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.094860 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.094985 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.097647 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:10.097651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:10.097651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:10.097662 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:10.099468 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.099880 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.099883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.099883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.099983 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.100217 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.100218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.100218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.100486 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:10.100620 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:10.100722 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:21:10.100861 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:21:10.126518 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39353
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:10.204838 14919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:10.205014 14919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:10.205046 14919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:10.206152 14919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:10.206189 14919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:10.206223 14919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:10.207688 14919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:10.207901 14919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:10.208102 14919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:10.212713 14928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:10.212834 14929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:10.212935 14919 server_base.cc:1034] running on GCE node
W20260809 19:21:10.213056 14932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:10.213271 14919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:10.213521 14919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:10.214663 14919 hybrid_clock.cc:648] HybridClock initialized: now 1786303270214653 us; error 31 us; skew 500 ppm
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:10.220224 14919 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:21:10.220641 14919 webserver.cc:460] Webserver started at http://127.9.91.65:34565/ using document root <none> and password file <none>
I20260809 19:21:10.220805 14919 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:10.220840 14919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:10.220927 14919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:10.221853 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "bbd6fa9671034d0d8b01adcb9e16a84e"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "21ca36663cfc73cef0b56f3b0bc28a0f"
server_key_iv: "c124e54b3fcf68613c72c39c49faeab7"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.222143 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "bbd6fa9671034d0d8b01adcb9e16a84e"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "21ca36663cfc73cef0b56f3b0bc28a0f"
server_key_iv: "c124e54b3fcf68613c72c39c49faeab7"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.223315 14919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:21:10.223908 14943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.224038 14919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:10.224125 14919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "bbd6fa9671034d0d8b01adcb9e16a84e"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "21ca36663cfc73cef0b56f3b0bc28a0f"
server_key_iv: "c124e54b3fcf68613c72c39c49faeab7"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.224190 14919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:10.231652 14919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:10.233180 14919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:10.233305 14919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:10.233494 14919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:10.233752 14919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:10.233783 14919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.233803 14919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:10.233824 14919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.239360 14919 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:43997
I20260809 19:21:10.239419 15095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:43997 every 8 connection(s)
I20260809 19:21:10.239673 14919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:10.240660 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 14919
I20260809 19:21:10.240749 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:10.240923 9581 external_mini_cluster.cc:1428] Setting key 0be01c4c16d659e4da9f451121e8a025
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:10.248703 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.240322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:60585 (local address 127.9.91.126:39353)
0809 19:21:10.240378 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:10.240380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:10.240540 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:21:10.241529 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:10.241531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:10.241543 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:10.241594 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:10.242155 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.242648 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.243950 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.244155 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.244215 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.246129 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:10.246130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:10.246130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:10.246138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:10.247406 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.247790 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.247791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.247791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.247862 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.248100 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.248101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.248102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.248241 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:10.248315 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:10.248521 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 19:21:10.248613 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:21:10.248733 15100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.240283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39353 (local address 127.9.91.65:60585)
0809 19:21:10.240423 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:10.240465 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:10.241430 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:10.241657 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:10.241661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:10.241892 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:10.242063 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:10.242067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.242729 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.242731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:10.243862 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:10.243866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.244238 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.244239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:10.244349 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.244784 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:10.244792 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:10.246044 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:10.247478 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:10.247485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:10.247487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:10.247698 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:10.247932 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:10.247933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:10.247933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:10.248027 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:10.248302 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:10.248305 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:10.248407 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 19:21:10.248477 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:21:10.248586 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260809 19:21:10.249300 15096 heartbeater.cc:347] Connected to a master server at 127.9.91.126:39353
I20260809 19:21:10.249390 15096 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
I20260809 19:21:10.249562 15096 heartbeater.cc:511] Master 127.9.91.126:39353 requested a full tablet report, sending...
I20260809 19:21:10.250056 14823 ts_manager.cc:194] Registered new tserver with Master: bbd6fa9671034d0d8b01adcb9e16a84e (127.9.91.65:43997)
I20260809 19:21:10.250667 14823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:60585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:21:10.265184 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39353
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:10.342545 15104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:10.342720 15104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:10.342742 15104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:10.343839 15104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:10.343873 15104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:10.343897 15104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:10.345299 15104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:10.345458 15104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:10.345619 15104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:10.350073 15111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:10.350119 15104 server_base.cc:1034] running on GCE node
W20260809 19:21:10.350143 15110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:10.350069 15113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:10.350452 15104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:10.350678 15104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:10.351811 15104 hybrid_clock.cc:648] HybridClock initialized: now 1786303270351800 us; error 24 us; skew 500 ppm
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:10.354051 15104 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:21:10.354437 15104 webserver.cc:460] Webserver started at http://127.9.91.66:33883/ using document root <none> and password file <none>
I20260809 19:21:10.354640 15104 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:10.354682 15104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:10.354802 15104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:10.355773 15104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "5eb29d4bf1604e4cbd47fe51efdcc198"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "c256cb79823f20afd5059a54b450e17f"
server_key_iv: "c0084e78c5a3928410d139649b8a85c5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.356078 15104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "5eb29d4bf1604e4cbd47fe51efdcc198"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "c256cb79823f20afd5059a54b450e17f"
server_key_iv: "c0084e78c5a3928410d139649b8a85c5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.357229 15104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:10.357820 15122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.357976 15104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:10.358054 15104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "5eb29d4bf1604e4cbd47fe51efdcc198"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "c256cb79823f20afd5059a54b450e17f"
server_key_iv: "c0084e78c5a3928410d139649b8a85c5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.358105 15104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:10.365147 15104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:10.367007 15104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:10.367108 15104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:10.367275 15104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:10.367498 15104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:10.367528 15104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.367547 15104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:10.367564 15104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.372870 15104 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:36435
I20260809 19:21:10.372936 15285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:36435 every 8 connection(s)
I20260809 19:21:10.373158 15104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:10.379328 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 15104
I20260809 19:21:10.379436 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:10.379618 9581 external_mini_cluster.cc:1428] Setting key e87ce153a8150a85ff2fb07e9e7acb55
I20260809 19:21:10.381271 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.373783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:51233 (local address 127.9.91.126:39353)
0809 19:21:10.373853 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:10.373855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:10.373980 (+ 125us) server_negotiation.cc:407] Connection header received
0809 19:21:10.374737 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:10.374738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:10.374747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:10.374790 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:10.375341 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.375816 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.376679 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.376857 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.376896 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.378649 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:10.378650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:10.378650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:10.378656 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:10.380087 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.380474 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.380476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.380477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.380542 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.380733 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.380733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.380733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.380856 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:10.380921 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:10.381108 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 19:21:10.381181 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:21:10.381336 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.373751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39353 (local address 127.9.91.66:51233)
0809 19:21:10.373875 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:10.373906 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:10.374651 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:10.374844 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:10.374847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:10.375109 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:10.375279 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:10.375283 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.375873 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.375873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:10.376622 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:10.376625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.376906 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.376907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:10.376992 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.377330 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:10.377337 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:10.378571 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:10.380162 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:10.380169 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:10.380171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:10.380406 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:10.380599 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:10.380600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:10.380600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:10.380685 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:10.380909 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:10.380912 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:10.381008 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 19:21:10.381078 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:21:10.381184 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260809 19:21:10.382256 15286 heartbeater.cc:347] Connected to a master server at 127.9.91.126:39353
I20260809 19:21:10.382333 15286 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:10.382493 15286 heartbeater.cc:511] Master 127.9.91.126:39353 requested a full tablet report, sending...
I20260809 19:21:10.382853 14823 ts_manager.cc:194] Registered new tserver with Master: 5eb29d4bf1604e4cbd47fe51efdcc198 (127.9.91.66:36435)
I20260809 19:21:10.383296 14823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:51233
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:21:10.403117 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39353
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:10.482515 15291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:10.482702 15291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:10.482722 15291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:10.483846 15291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:10.483889 15291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:10.483913 15291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:10.485383 15291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:10.485598 15291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:10.485803 15291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:10.490749 15300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:10.490756 15303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:10.490808 15299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:10.490999 15291 server_base.cc:1034] running on GCE node
I20260809 19:21:10.491214 15291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:10.491430 15291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:10.492568 15291 hybrid_clock.cc:648] HybridClock initialized: now 1786303270492555 us; error 31 us; skew 500 ppm
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:10.494870 15291 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:21:10.495270 15291 webserver.cc:460] Webserver started at http://127.9.91.67:35387/ using document root <none> and password file <none>
I20260809 19:21:10.495456 15291 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:10.495501 15291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:10.495625 15291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:10.496582 15291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "2cd7ac0edb8f4788a874e36e6a7b7e97"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "cbbe9efa06f94719cc8328b1332cffdf"
server_key_iv: "c43d64f059559bc3e719be4cc83947e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.496934 15291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd7ac0edb8f4788a874e36e6a7b7e97"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "cbbe9efa06f94719cc8328b1332cffdf"
server_key_iv: "c43d64f059559bc3e719be4cc83947e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.498062 15291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:21:10.498689 15316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.498832 15291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.498934 15291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "2cd7ac0edb8f4788a874e36e6a7b7e97"
format_stamp: "Formatted at 2026-08-09 19:21:10 on dist-test-slave-1vmg"
server_key: "cbbe9efa06f94719cc8328b1332cffdf"
server_key_iv: "c43d64f059559bc3e719be4cc83947e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:10.499011 15291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:10.515625 15291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:10.517315 15291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:10.517447 15291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:10.517650 15291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:10.517947 15291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:10.517982 15291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.518008 15291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:10.518033 15291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:10.523805 15291 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:46431
I20260809 19:21:10.523844 15484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:46431 every 8 connection(s)
I20260809 19:21:10.524130 15291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:10.527235 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 15291
I20260809 19:21:10.527343 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:10.527522 9581 external_mini_cluster.cc:1428] Setting key e194b4d02cd36d33e6a9029b1906d5f5
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:10.532536 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.524803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:42515 (local address 127.9.91.126:39353)
0809 19:21:10.524865 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:10.524867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:10.525038 (+ 171us) server_negotiation.cc:407] Connection header received
0809 19:21:10.525891 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:10.525892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:10.525901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:10.525954 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:10.526499 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.527107 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.528407 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.528619 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.528668 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.530497 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:10.530499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:10.530499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:10.530505 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:10.531570 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.531896 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.531896 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.531897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.531939 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.532103 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.532104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.532104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.532189 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:10.532229 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:10.532421 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 19:21:10.532474 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:21:10.532699 15488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.524761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39353 (local address 127.9.91.67:42515)
0809 19:21:10.524924 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:10.524964 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:10.525815 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:10.526006 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:10.526009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:10.526250 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:10.526436 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:10.526440 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.527193 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.527194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:10.528330 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:10.528333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.528676 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.528677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:10.528777 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.529155 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:10.529162 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:10.530414 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:10.531632 (+ 1218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:10.531639 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:10.531640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:10.531842 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:10.531980 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:10.531981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:10.531981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:10.532064 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:10.532228 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:10.532231 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:10.532331 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 19:21:10.532402 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:21:10.532509 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260809 19:21:10.533092 15485 heartbeater.cc:347] Connected to a master server at 127.9.91.126:39353
I20260809 19:21:10.533167 15485 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:10.533337 15485 heartbeater.cc:511] Master 127.9.91.126:39353 requested a full tablet report, sending...
I20260809 19:21:10.533663 14823 ts_manager.cc:194] Registered new tserver with Master: 2cd7ac0edb8f4788a874e36e6a7b7e97 (127.9.91.67:46431)
I20260809 19:21:10.533990 14823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:42515
I20260809 19:21:10.539018 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:21:10.561697 15494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.554953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.9.91.65:43997)
0809 19:21:10.555195 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:10.555199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:10.555209 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:10.555231 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:10.555232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:10.555245 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:10.555305 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:10.555792 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.556454 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.557098 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.557376 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.557421 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.559361 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:10.559370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:10.559371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:10.559378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:10.560641 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.560943 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.560946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.560946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.561005 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.561176 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.561177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.561177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.561313 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:10.561437 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:10.561515 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 19:21:10.561594 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":155,"threads_started":1}
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[14758](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303270, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:10.568305 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:10.562867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.9.91.126:39353)
0809 19:21:10.562953 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:10.562955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:10.562964 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:10.563005 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:10.563005 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:10.563013 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:10.563074 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:10.563454 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.563885 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.564449 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:10.564616 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:10.564660 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:10.566446 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:10.566448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:10.566448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:10.566454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:10.567526 (+ 1072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.567789 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.567789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.567790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.567830 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:10.567954 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:10.567955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:10.567955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:10.568067 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:10.568127 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:10.568197 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 19:21:10.568248 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 19:21:10.568509 14823 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43616
I20260809 19:21:10.568787 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 14919
I20260809 19:21:10.585244 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 15104
I20260809 19:21:10.614071 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 15291
I20260809 19:21:10.623437 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 14776
2026-08-09T19:21:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1881 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[15499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[15499](info): set up 2 sockets
Aug 09 19:21:10 dist-test-slave-1vmg krb5kdc[15499](info): commencing operation
krb5kdc: starting...
W20260809 19:21:11.750417 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:11 dist-test-slave-1vmg krb5kdc[15499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:11.855561 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43343
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:43343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:11.933750 15519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:11.933921 15519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:11.933952 15519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:11.935068 15519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:11.935110 15519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:11.935132 15519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:11.935146 15519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:11.935163 15519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:11.936599 15519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:43343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:11.936818 15519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:11.937000 15519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:11.941303 15531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:11.941393 15529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:11.941478 15528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:11.941486 15519 server_base.cc:1034] running on GCE node
I20260809 19:21:11.941757 15519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:11.942008 15519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:11.943140 15519 hybrid_clock.cc:648] HybridClock initialized: now 1786303271943123 us; error 29 us; skew 500 ppm
Aug 09 19:21:11 dist-test-slave-1vmg krb5kdc[15499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303271, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:11.945427 15519 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:11.945716 15519 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:11.945761 15519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:11.945848 15519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:11.946784 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "f725c51503ad436ba7f63c9ecc74f9a2"
format_stamp: "Formatted at 2026-08-09 19:21:11 on dist-test-slave-1vmg"
server_key: "93ded29ba60c7a40c25ff6bd2041e1b1"
server_key_iv: "2aa790ca65e1f0d3ab78a87401dbb911"
server_key_version: "encryptionkey@0"
I20260809 19:21:11.947098 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "f725c51503ad436ba7f63c9ecc74f9a2"
format_stamp: "Formatted at 2026-08-09 19:21:11 on dist-test-slave-1vmg"
server_key: "93ded29ba60c7a40c25ff6bd2041e1b1"
server_key_iv: "2aa790ca65e1f0d3ab78a87401dbb911"
server_key_version: "encryptionkey@0"
I20260809 19:21:11.948185 15519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:11.948758 15538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:11.948923 15519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:11.949002 15519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "f725c51503ad436ba7f63c9ecc74f9a2"
format_stamp: "Formatted at 2026-08-09 19:21:11 on dist-test-slave-1vmg"
server_key: "93ded29ba60c7a40c25ff6bd2041e1b1"
server_key_iv: "2aa790ca65e1f0d3ab78a87401dbb911"
server_key_version: "encryptionkey@0"
I20260809 19:21:11.949051 15519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:11.968508 15519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:11.970022 15519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:11.970120 15519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:11.973626 15519 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:43343
I20260809 19:21:11.973654 15616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:43343 every 8 connection(s)
I20260809 19:21:11.973883 15519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:11.974781 15619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:11.976792 15619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2: Bootstrap starting.
I20260809 19:21:11.977327 15619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:11.977607 15619 log.cc:824] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:11.978202 15619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2: No bootstrap required, opened a new log
I20260809 19:21:11.979321 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 15519
I20260809 19:21:11.979425 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:11.979616 9581 external_mini_cluster.cc:1428] Setting key b9f4f8b18c26506ae875dc970a6bcb9b
I20260809 19:21:11.979952 15619 raft_consensus.cc:348] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43343 } }
I20260809 19:21:11.980082 15619 raft_consensus.cc:374] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:11.980157 15619 raft_consensus.cc:729] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f725c51503ad436ba7f63c9ecc74f9a2, State: Initialized, Role: FOLLOWER
I20260809 19:21:11.980311 15619 consensus_queue.cc:260] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43343 } }
I20260809 19:21:11.980381 15619 raft_consensus.cc:388] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:11.980412 15619 raft_consensus.cc:482] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:11.980456 15619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:11.981231 15619 raft_consensus.cc:504] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43343 } }
I20260809 19:21:11.981324 15619 leader_election.cc:302] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f725c51503ad436ba7f63c9ecc74f9a2; no voters:
I20260809 19:21:11.981469 15619 leader_election.cc:288] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:11.981508 15626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:11.981652 15626 raft_consensus.cc:686] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [term 1 LEADER]: Becoming Leader. State: Replica: f725c51503ad436ba7f63c9ecc74f9a2, State: Running, Role: LEADER
I20260809 19:21:11.981717 15619 sys_catalog.cc:564] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:11.981773 15626 consensus_queue.cc:237] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43343 } }
I20260809 19:21:11.982152 15626 sys_catalog.cc:455] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f725c51503ad436ba7f63c9ecc74f9a2. Latest consensus state: current_term: 1 leader_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43343 } } }
I20260809 19:21:11.982159 15629 sys_catalog.cc:455] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f725c51503ad436ba7f63c9ecc74f9a2" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43343 } } }
I20260809 19:21:11.982259 15626 sys_catalog.cc:458] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:11.982270 15629 sys_catalog.cc:458] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2 [sys.catalog]: This master's current role is: LEADER
W20260809 19:21:11.983678 15652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:21:11.983737 15652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:21:11.983788 15636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:11.983923 15636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:21:11 dist-test-slave-1vmg krb5kdc[15499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:11.986553 15636 catalog_manager.cc:1347] Generated new cluster ID: 323df36bf6334db2bd8e142f9404a535
I20260809 19:21:11.986594 15636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:11.990438 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:11.980143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.9.91.126:43343)
0809 19:21:11.980342 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:11.980350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:11.980378 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:21:11.980478 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:11.980484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:11.980536 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:11.980657 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:11.981253 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:11.982532 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:11.983564 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:11.984070 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:11.984160 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:11.986643 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:11.986647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:11.986647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:11.986658 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:11.988432 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:11.988826 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:11.988828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:11.988828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:11.989400 (+ 572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:11.989632 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:11.989633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:11.989633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:11.989893 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:11.990023 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:11.990133 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 19:21:11.990258 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260809 19:21:12.011593 15636 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:12.012282 15636 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:12.023036 15636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2: Generated new TSK 0
I20260809 19:21:12.023197 15636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:21:12.053150 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43343
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:12.131753 15658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:12.131940 15658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:12.131973 15658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:12.133071 15658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:12.133111 15658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:12.133147 15658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:12.134574 15658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:12.134799 15658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:12.135015 15658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:12.139506 15669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:12.139494 15666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:12.139492 15664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.139799 15658 server_base.cc:1034] running on GCE node
I20260809 19:21:12.139993 15658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:12.140172 15658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:12.141299 15658 hybrid_clock.cc:648] HybridClock initialized: now 1786303272141291 us; error 27 us; skew 500 ppm
Aug 09 19:21:12 dist-test-slave-1vmg krb5kdc[15499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303272, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:12.143640 15658 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:21:12.143958 15658 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:12.144008 15658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:12.144115 15658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:12.145061 15658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "2971a241def04b5785b8c875d7783813"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "d5d9e61d33406122e7cc6ad4f215f530"
server_key_iv: "8724b93248d6fba8fc86054258dc80b8"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.145375 15658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "2971a241def04b5785b8c875d7783813"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "d5d9e61d33406122e7cc6ad4f215f530"
server_key_iv: "8724b93248d6fba8fc86054258dc80b8"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.146620 15658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:12.147290 15679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.147430 15658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.147507 15658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "2971a241def04b5785b8c875d7783813"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "d5d9e61d33406122e7cc6ad4f215f530"
server_key_iv: "8724b93248d6fba8fc86054258dc80b8"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.147569 15658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:12.162014 15658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:12.163740 15658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:12.163872 15658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:12.164069 15658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:12.164340 15658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:12.164371 15658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.164390 15658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:12.164413 15658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.169732 15658 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:41393
I20260809 19:21:12.169766 15855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:41393 every 8 connection(s)
I20260809 19:21:12.169991 15658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:21:12 dist-test-slave-1vmg krb5kdc[15499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303272, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:12.177418 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 15658
I20260809 19:21:12.177542 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:12.177744 9581 external_mini_cluster.cc:1428] Setting key fff3cc37196a4b08cde640fed83fdf1a
I20260809 19:21:12.178407 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.170614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:58107 (local address 127.9.91.126:43343)
0809 19:21:12.170673 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.170675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.170847 (+ 172us) server_negotiation.cc:407] Connection header received
0809 19:21:12.171315 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.171315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.171329 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.171378 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:12.172330 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.172826 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.173724 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.173922 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.173966 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.175809 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:12.175811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:12.175811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:12.175818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:12.177101 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.177437 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.177438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.177439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.177508 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.177754 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.177755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.177755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.177896 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:12.177957 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.178243 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.178332 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:21:12.178583 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.170573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43343 (local address 127.9.91.65:58107)
0809 19:21:12.170734 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.170772 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.171236 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.171428 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.171431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.171667 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:12.172245 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.172251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.172888 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.172889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.173663 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.173665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.173974 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.173975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.174062 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.174437 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:12.174445 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:12.175723 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:12.177159 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:12.177163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:12.177164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:12.177376 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:12.177574 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:12.177575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:12.177576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:12.177694 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:12.177960 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:12.177967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:12.178117 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.178223 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.178373 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260809 19:21:12.179036 15856 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43343
I20260809 19:21:12.179136 15856 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:12.179282 15856 heartbeater.cc:511] Master 127.9.91.126:43343 requested a full tablet report, sending...
I20260809 19:21:12.179749 15564 ts_manager.cc:194] Registered new tserver with Master: 2971a241def04b5785b8c875d7783813 (127.9.91.65:41393)
I20260809 19:21:12.180517 15564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:58107
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:21:12.201109 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43343
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:12.281013 15862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:12.281203 15862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:12.281243 15862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:12.282419 15862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:12.282466 15862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:12.282497 15862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:12.284060 15862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:12.284263 15862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:12.284461 15862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:12.289108 15869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:12.289083 15867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.289149 15862 server_base.cc:1034] running on GCE node
W20260809 19:21:12.289064 15878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.289498 15862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:12.289722 15862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:12.290867 15862 hybrid_clock.cc:648] HybridClock initialized: now 1786303272290852 us; error 28 us; skew 500 ppm
Aug 09 19:21:12 dist-test-slave-1vmg krb5kdc[15499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303272, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:12.293260 15862 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:21:12.293527 15862 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:12.293568 15862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:12.293663 15862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:12.294564 15862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "5d8d91271ebc4291aa0b62f999767129"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "3a55a8d3d9744d05a76a2d47adc6163c"
server_key_iv: "97965c4d4baccae1f32858e5803dd0ce"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.294847 15862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "5d8d91271ebc4291aa0b62f999767129"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "3a55a8d3d9744d05a76a2d47adc6163c"
server_key_iv: "97965c4d4baccae1f32858e5803dd0ce"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.296169 15862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:12.296824 15889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.297034 15862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:12.297123 15862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "5d8d91271ebc4291aa0b62f999767129"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "3a55a8d3d9744d05a76a2d47adc6163c"
server_key_iv: "97965c4d4baccae1f32858e5803dd0ce"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.297206 15862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:12.304543 15862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:12.306200 15862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:12.306331 15862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:12.306535 15862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:12.306820 15862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:12.306856 15862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.306879 15862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:12.306908 15862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.312917 15862 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:41789
I20260809 19:21:12.312973 16059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:41789 every 8 connection(s)
I20260809 19:21:12.313200 15862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:12.315289 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 15862
I20260809 19:21:12.315411 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:12.315614 9581 external_mini_cluster.cc:1428] Setting key 107f82f9f35e672f8d40076d87ec3c16
Aug 09 19:21:12 dist-test-slave-1vmg krb5kdc[15499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303272, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:12.322636 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.313843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:56805 (local address 127.9.91.126:43343)
0809 19:21:12.313883 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.313885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.314069 (+ 184us) server_negotiation.cc:407] Connection header received
0809 19:21:12.314512 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.314514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.314528 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.314581 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:12.315606 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.316317 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.317569 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.317762 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.317805 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.319916 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:12.319917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:12.319917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:12.319923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:12.321399 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.321766 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.321767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.321768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.321835 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.322033 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.322033 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.322034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.322173 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:12.322251 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.322450 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.322548 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:12.322669 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.313829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43343 (local address 127.9.91.66:56805)
0809 19:21:12.313966 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.313996 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.314421 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.314640 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.314644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.314892 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:12.315508 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.315515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.316407 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.316408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.317497 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.317500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.317824 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.317825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.317953 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.318412 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:12.318420 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:12.319817 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:12.321475 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:12.321481 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:12.321483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:12.321696 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:12.321895 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:12.321895 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:12.321895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:12.321980 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:12.322239 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:12.322241 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:12.322339 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.322411 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.322516 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260809 19:21:12.323122 16060 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43343
I20260809 19:21:12.323202 16060 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:12.323328 16060 heartbeater.cc:511] Master 127.9.91.126:43343 requested a full tablet report, sending...
I20260809 19:21:12.323673 15564 ts_manager.cc:194] Registered new tserver with Master: 5d8d91271ebc4291aa0b62f999767129 (127.9.91.66:41789)
I20260809 19:21:12.324023 15564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:56805
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:21:12.339720 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43343
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:12.418274 16071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:12.418450 16071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:12.418471 16071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:12.419602 16071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:12.419642 16071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:12.419672 16071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:12.421227 16071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:12.421442 16071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:12.421640 16071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:12.426306 16080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:12.426299 16077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.426373 16071 server_base.cc:1034] running on GCE node
W20260809 19:21:12.426287 16078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.426684 16071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:12.426928 16071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:12.428064 16071 hybrid_clock.cc:648] HybridClock initialized: now 1786303272428051 us; error 33 us; skew 500 ppm
Aug 09 19:21:12 dist-test-slave-1vmg krb5kdc[15499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303272, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:12.430389 16071 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:21:12.430722 16071 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:12.430770 16071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:12.430891 16071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:12.431851 16071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "134918b3af6f458781320b06b62015df"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "b99c99d133b45fe2fd1c251592b8448b"
server_key_iv: "925ca73c8eaa6214941e25829a14f6a2"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.432158 16071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "134918b3af6f458781320b06b62015df"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "b99c99d133b45fe2fd1c251592b8448b"
server_key_iv: "925ca73c8eaa6214941e25829a14f6a2"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.433326 16071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:21:12.433951 16092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.434095 16071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:12.434170 16071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "134918b3af6f458781320b06b62015df"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "b99c99d133b45fe2fd1c251592b8448b"
server_key_iv: "925ca73c8eaa6214941e25829a14f6a2"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.434243 16071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:12.444298 16071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:12.445858 16071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:12.445971 16071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:12.446153 16071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:12.446436 16071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:12.446470 16071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.446488 16071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:12.446511 16071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.452528 16071 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:36811
I20260809 19:21:12.452612 16259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:36811 every 8 connection(s)
I20260809 19:21:12.452874 16071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:12.453688 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 16071
I20260809 19:21:12.453802 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:12.454010 9581 external_mini_cluster.cc:1428] Setting key 93b6b3fb199e75c8d7360f3fb8926ea1
Aug 09 19:21:12 dist-test-slave-1vmg krb5kdc[15499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303272, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:12.461205 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.453508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:55139 (local address 127.9.91.126:43343)
0809 19:21:12.453584 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.453586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.453777 (+ 191us) server_negotiation.cc:407] Connection header received
0809 19:21:12.454383 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.454384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.454395 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.454453 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:12.455564 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.456055 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.456968 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.457138 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.457178 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.459171 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:12.459172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:12.459173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:12.459178 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:12.460233 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.460550 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.460550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.460551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.460594 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.460761 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.460761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.460762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.460851 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:12.460894 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.461079 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.461135 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:21:12.461367 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.453491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43343 (local address 127.9.91.67:55139)
0809 19:21:12.453639 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.453686 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.454289 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.454527 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.454531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.454945 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:12.455494 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.455500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.456125 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.456126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.456909 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.456911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.457190 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.457191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.457282 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.457754 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:12.457763 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:12.459103 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:12.460294 (+ 1191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:12.460299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:12.460301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:12.460495 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:12.460635 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:12.460636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:12.460636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:12.460721 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:12.460892 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:12.460894 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:12.460995 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.461065 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.461178 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260809 19:21:12.461826 16260 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43343
I20260809 19:21:12.461902 16260 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:12.462013 16260 heartbeater.cc:511] Master 127.9.91.126:43343 requested a full tablet report, sending...
I20260809 19:21:12.462298 15564 ts_manager.cc:194] Registered new tserver with Master: 134918b3af6f458781320b06b62015df (127.9.91.67:36811)
I20260809 19:21:12.462651 15564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:55139
I20260809 19:21:12.465561 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:21:12.471657 15625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.466752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.9.91.126:43343)
0809 19:21:12.466787 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.466788 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.466889 (+ 101us) server_negotiation.cc:407] Connection header received
0809 19:21:12.466979 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.466980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.466987 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.467030 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:12.467520 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.467962 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.468607 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.468776 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.468815 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.469787 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:12.469787 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:12.469788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:12.469791 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:12.470770 (+ 979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.471120 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.471120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.471121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.471166 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:12.471296 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:12.471296 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:12.471297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:12.471411 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:12.471478 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.471546 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.471602 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:21:12.472944 15564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41566:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:12.474407 15564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:12.480286 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.476465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54504 (local address 127.9.91.65:41393)
0809 19:21:12.476674 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.476678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.476688 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:12.476721 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.476722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.476733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.476800 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:12.476935 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.477766 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.479276 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.479807 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.479846 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.479885 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.480141 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.480198 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":81,"threads_started":1}
I20260809 19:21:12.480281 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.476320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41789 (local address 127.0.0.1:43242)
0809 19:21:12.476458 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.476497 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.476555 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.477175 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.477176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.477178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:12.477221 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.477223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.478252 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.478253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.479026 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.479029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.479974 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.479975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.480031 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.480038 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.480107 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.480173 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":1,"thread_start_us":55,"threads_started":1}
I20260809 19:21:12.480280 16271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.476328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:41393 (local address 127.0.0.1:54504)
0809 19:21:12.476506 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.476517 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.476573 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.476813 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.476816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.476819 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:12.476879 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.476881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.477858 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.477860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.479168 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.479171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.479935 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.479936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.480024 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.480037 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.480118 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.480174 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":23,"thread_start_us":146,"threads_started":1}
I20260809 19:21:12.480711 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.476785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.9.91.66:41789)
0809 19:21:12.477002 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.477006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.477015 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:12.477055 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.477057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.477071 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.477297 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:12.477385 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.478140 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.479122 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.479886 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.479958 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.480010 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.480579 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.480627 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":135,"threads_started":1}
I20260809 19:21:12.481207 15758 tablet_service.cc:1467] Processing CreateTablet for tablet a71d18f42ef6457a832e2eb13f1099da (DEFAULT_TABLE table=test-table [id=eee6095056b74f10af3f6a27223cee05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:12.481427 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.476574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:36811 (local address 127.0.0.1:34328)
0809 19:21:12.476753 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.476763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.476810 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.478065 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.478066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.478068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:12.478115 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.478116 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.479314 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.479314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.480356 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.480359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.481204 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.481205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.481266 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.481271 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.481319 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.481359 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":128,"threads_started":1}
I20260809 19:21:12.481374 15966 tablet_service.cc:1467] Processing CreateTablet for tablet a71d18f42ef6457a832e2eb13f1099da (DEFAULT_TABLE table=test-table [id=eee6095056b74f10af3f6a27223cee05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:12.481635 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.477100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.9.91.67:36811)
0809 19:21:12.477913 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.477917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.477925 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:12.477954 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.477956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.477970 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.478045 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:12.478192 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.479234 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.480452 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.481119 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.481182 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.481237 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.481515 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.481556 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":83,"threads_started":1}
I20260809 19:21:12.481643 15758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71d18f42ef6457a832e2eb13f1099da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:12.481691 15966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71d18f42ef6457a832e2eb13f1099da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:12.482219 16161 tablet_service.cc:1467] Processing CreateTablet for tablet a71d18f42ef6457a832e2eb13f1099da (DEFAULT_TABLE table=test-table [id=eee6095056b74f10af3f6a27223cee05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:12.482477 16161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a71d18f42ef6457a832e2eb13f1099da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:12.485127 16277 tablet_bootstrap.cc:492] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Bootstrap starting.
I20260809 19:21:12.485543 16281 tablet_bootstrap.cc:492] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Bootstrap starting.
I20260809 19:21:12.485859 16277 tablet_bootstrap.cc:654] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:12.486140 16281 tablet_bootstrap.cc:654] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:12.486361 16277 log.cc:824] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:12.486665 16280 tablet_bootstrap.cc:492] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813: Bootstrap starting.
I20260809 19:21:12.486686 16281 log.cc:824] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:12.487293 16280 tablet_bootstrap.cc:654] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:12.487419 16281 tablet_bootstrap.cc:492] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: No bootstrap required, opened a new log
I20260809 19:21:12.487473 16281 ts_tablet_manager.cc:1397] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:12.487880 16280 log.cc:824] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:12.488323 16277 tablet_bootstrap.cc:492] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: No bootstrap required, opened a new log
I20260809 19:21:12.488394 16277 ts_tablet_manager.cc:1397] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:21:12.488647 16280 tablet_bootstrap.cc:492] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813: No bootstrap required, opened a new log
I20260809 19:21:12.488715 16280 ts_tablet_manager.cc:1397] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:12.489153 16281 raft_consensus.cc:348] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.489310 16281 raft_consensus.cc:374] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:12.489343 16281 raft_consensus.cc:729] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134918b3af6f458781320b06b62015df, State: Initialized, Role: FOLLOWER
I20260809 19:21:12.489447 16281 consensus_queue.cc:260] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.490352 16277 raft_consensus.cc:348] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.490499 16277 raft_consensus.cc:374] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:12.490526 16277 raft_consensus.cc:729] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8d91271ebc4291aa0b62f999767129, State: Initialized, Role: FOLLOWER
I20260809 19:21:12.490653 16277 consensus_queue.cc:260] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.490662 16280 raft_consensus.cc:348] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.490799 16280 raft_consensus.cc:374] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:12.490831 16280 raft_consensus.cc:729] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2971a241def04b5785b8c875d7783813, State: Initialized, Role: FOLLOWER
I20260809 19:21:12.490938 16280 consensus_queue.cc:260] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.491412 16280 ts_tablet_manager.cc:1428] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:21:12.491453 16277 ts_tablet_manager.cc:1428] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:21:12.491565 16060 heartbeater.cc:503] Master 127.9.91.126:43343 was elected leader, sending a full tablet report...
I20260809 19:21:12.491578 15856 heartbeater.cc:503] Master 127.9.91.126:43343 was elected leader, sending a full tablet report...
I20260809 19:21:12.491569 16281 ts_tablet_manager.cc:1428] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 19:21:12.491724 16260 heartbeater.cc:503] Master 127.9.91.126:43343 was elected leader, sending a full tablet report...
I20260809 19:21:12.545105 16291 raft_consensus.cc:482] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:12.545244 16291 raft_consensus.cc:504] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.545621 16291 leader_election.cc:288] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d8d91271ebc4291aa0b62f999767129 (127.9.91.66:41789), 134918b3af6f458781320b06b62015df (127.9.91.67:36811)
I20260809 19:21:12.548318 16275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.545875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:37795 (local address 127.9.91.66:41789)
0809 19:21:12.545941 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.545943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.545952 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:12.545982 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.545983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.545994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.546066 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:12.546185 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.546659 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.547655 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.548004 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.548093 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.548104 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.548228 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.548260 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:21:12.548311 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.545785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41789 (local address 127.9.91.65:37795)
0809 19:21:12.545847 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.545859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.545897 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.546071 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.546072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.546074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:12.546128 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.546130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.546741 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.546742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.547576 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.547578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.548093 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.548094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.548160 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.548164 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.548204 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.548240 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 19:21:12.548563 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.545949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:60021 (local address 127.9.91.67:36811)
0809 19:21:12.545999 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.546001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.546100 (+ 99us) server_negotiation.cc:407] Connection header received
0809 19:21:12.546167 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.546168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.546181 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.546228 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:12.546378 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.546836 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.547776 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.548177 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.548244 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.548254 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.548450 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.548490 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:21:12.548704 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d18f42ef6457a832e2eb13f1099da" candidate_uuid: "2971a241def04b5785b8c875d7783813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d8d91271ebc4291aa0b62f999767129" is_pre_election: true
I20260809 19:21:12.548764 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.545819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:36811 (local address 127.9.91.65:60021)
0809 19:21:12.546029 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.546039 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.546098 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.546238 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.546239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.546241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:12.546318 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.546319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.546938 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.546939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.547690 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:12.547692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.548288 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.548289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:12.548360 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.548364 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.548653 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.548695 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":121,"threads_started":1}
I20260809 19:21:12.548866 15998 raft_consensus.cc:2454] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2971a241def04b5785b8c875d7783813 in term 0.
I20260809 19:21:12.549106 15682 leader_election.cc:302] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2971a241def04b5785b8c875d7783813, 5d8d91271ebc4291aa0b62f999767129; no voters:
I20260809 19:21:12.549098 16188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d18f42ef6457a832e2eb13f1099da" candidate_uuid: "2971a241def04b5785b8c875d7783813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "134918b3af6f458781320b06b62015df" is_pre_election: true
I20260809 19:21:12.549220 16188 raft_consensus.cc:2454] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2971a241def04b5785b8c875d7783813 in term 0.
I20260809 19:21:12.549329 16291 raft_consensus.cc:2781] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:12.549405 16291 raft_consensus.cc:482] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:12.549435 16291 raft_consensus.cc:3037] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:12.550105 16291 raft_consensus.cc:504] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.550225 16291 leader_election.cc:288] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [CANDIDATE]: Term 1 election: Requested vote from peers 5d8d91271ebc4291aa0b62f999767129 (127.9.91.66:41789), 134918b3af6f458781320b06b62015df (127.9.91.67:36811)
I20260809 19:21:12.550395 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d18f42ef6457a832e2eb13f1099da" candidate_uuid: "2971a241def04b5785b8c875d7783813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d8d91271ebc4291aa0b62f999767129"
I20260809 19:21:12.550444 16188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a71d18f42ef6457a832e2eb13f1099da" candidate_uuid: "2971a241def04b5785b8c875d7783813" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "134918b3af6f458781320b06b62015df"
I20260809 19:21:12.550468 15998 raft_consensus.cc:3037] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:12.550524 16188 raft_consensus.cc:3037] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:12.551270 15998 raft_consensus.cc:2454] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2971a241def04b5785b8c875d7783813 in term 1.
I20260809 19:21:12.551360 16188 raft_consensus.cc:2454] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2971a241def04b5785b8c875d7783813 in term 1.
I20260809 19:21:12.551456 15682 leader_election.cc:302] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2971a241def04b5785b8c875d7783813, 5d8d91271ebc4291aa0b62f999767129; no voters:
I20260809 19:21:12.551550 16291 raft_consensus.cc:2781] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:12.551708 16291 raft_consensus.cc:686] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 1 LEADER]: Becoming Leader. State: Replica: 2971a241def04b5785b8c875d7783813, State: Running, Role: LEADER
I20260809 19:21:12.551802 16291 consensus_queue.cc:237] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } }
I20260809 19:21:12.552596 15558 catalog_manager.cc:5495] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2971a241def04b5785b8c875d7783813 (127.9.91.65). New cstate: current_term: 1 leader_uuid: "2971a241def04b5785b8c875d7783813" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2971a241def04b5785b8c875d7783813" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 41393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 } health_report { overall_health: UNKNOWN } } }
W20260809 19:21:12.563659 16062 tablet.cc:2369] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:21:12.570474 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.568576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.9.91.65:41393)
0809 19:21:12.568635 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.568636 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.568643 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:12.568668 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.568669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.568684 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.568738 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:12.568862 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.569286 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.569931 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:12.570097 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:12.570135 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:12.570237 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:12.570323 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:12.570373 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.570388 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.570402 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:21:12.572054 16188 raft_consensus.cc:1261] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 1 FOLLOWER]: Refusing update from remote peer 2971a241def04b5785b8c875d7783813: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:12.572072 15998 raft_consensus.cc:1261] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 1 FOLLOWER]: Refusing update from remote peer 2971a241def04b5785b8c875d7783813: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:12.572365 16293 consensus_queue.cc:1035] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "134918b3af6f458781320b06b62015df" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:12.572484 16291 consensus_queue.cc:1035] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8d91271ebc4291aa0b62f999767129" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:12.574739 16297 mvcc.cc:205] Tried to move back new op lower bound from 7316698204452904960 to 7316698204372824064. Current Snapshot: MvccSnapshot[applied={T|T < 7316698204452904960}]
I20260809 19:21:12.577462 15558 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:12.577538 15558 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41566:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:12.578192 15558 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f725c51503ad436ba7f63c9ecc74f9a2: Sending DeleteTablet for 3 replicas of tablet a71d18f42ef6457a832e2eb13f1099da
I20260809 19:21:12.578536 15758 tablet_service.cc:1514] Processing DeleteTablet for tablet a71d18f42ef6457a832e2eb13f1099da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:12 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:54504
I20260809 19:21:12.578532 15966 tablet_service.cc:1514] Processing DeleteTablet for tablet a71d18f42ef6457a832e2eb13f1099da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:12 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:43242
I20260809 19:21:12.578743 16161 tablet_service.cc:1514] Processing DeleteTablet for tablet a71d18f42ef6457a832e2eb13f1099da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:12 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:34328
I20260809 19:21:12.578809 16312 tablet_replica.cc:330] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813: stopping tablet replica
I20260809 19:21:12.578964 16314 tablet_replica.cc:330] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: stopping tablet replica
I20260809 19:21:12.578998 16313 tablet_replica.cc:330] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: stopping tablet replica
I20260809 19:21:12.579052 16314 raft_consensus.cc:2229] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:12.579074 16313 raft_consensus.cc:2229] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:12.579124 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 15658
I20260809 19:21:12.579149 16314 raft_consensus.cc:2258] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:12.579161 16313 raft_consensus.cc:2258] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:12.578899 16312 raft_consensus.cc:2229] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:12.579331 16312 raft_consensus.cc:2258] T a71d18f42ef6457a832e2eb13f1099da P 2971a241def04b5785b8c875d7783813 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:12.583420 16313 ts_tablet_manager.cc:1905] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:12.585886 16313 ts_tablet_manager.cc:1918] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:12.587437 16313 log.cc:1196] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/a71d18f42ef6457a832e2eb13f1099da
I20260809 19:21:12.587909 16313 ts_tablet_manager.cc:1939] T a71d18f42ef6457a832e2eb13f1099da P 5d8d91271ebc4291aa0b62f999767129: Deleting consensus metadata
I20260809 19:21:12.588415 15543 catalog_manager.cc:4841] TS 5d8d91271ebc4291aa0b62f999767129 (127.9.91.66:41789): tablet a71d18f42ef6457a832e2eb13f1099da (table test-table [id=eee6095056b74f10af3f6a27223cee05]) successfully deleted
I20260809 19:21:12.588973 16314 ts_tablet_manager.cc:1905] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:12.590214 16314 ts_tablet_manager.cc:1918] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:12.590272 16314 log.cc:1196] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/a71d18f42ef6457a832e2eb13f1099da
I20260809 19:21:12.590480 16314 ts_tablet_manager.cc:1939] T a71d18f42ef6457a832e2eb13f1099da P 134918b3af6f458781320b06b62015df: Deleting consensus metadata
I20260809 19:21:12.590864 15544 catalog_manager.cc:4841] TS 134918b3af6f458781320b06b62015df (127.9.91.67:36811): tablet a71d18f42ef6457a832e2eb13f1099da (table test-table [id=eee6095056b74f10af3f6a27223cee05]) successfully deleted
W20260809 19:21:12.594231 15544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:41393 (error 108)
I20260809 19:21:12.594581 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 15862
I20260809 19:21:12.594722 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.594585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:41393 (local address 127.0.0.1:54526)
0809 19:21:12.594644 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.594689 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:41393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 19:21:12.598093 15544 catalog_manager.cc:4584] TS 2971a241def04b5785b8c875d7783813 (127.9.91.65:41393): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a71d18f42ef6457a832e2eb13f1099da: Network error: Client connection negotiation failed: client connection to 127.9.91.65:41393: connect: Connection refused (error 111)
I20260809 19:21:12.602512 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 16071
I20260809 19:21:12.613940 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 15519
2026-08-09T19:21:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1993 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T19:21:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:12Z Disabled control of system clock
I20260809 19:21:12.646946 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:41253
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:38471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:41253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:21:12.726262 16320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:12.726462 16320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:12.726485 16320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:12.727638 16320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:12.727679 16320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:12.727689 16320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:12.727700 16320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:12.727711 16320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:12.728994 16320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:41253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:41253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:12.729176 16320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:12.729403 16320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:12.733731 16329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:12.733757 16332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:12.733944 16330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.733994 16320 server_base.cc:1034] running on GCE node
I20260809 19:21:12.734222 16320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:12.734514 16320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:12.735647 16320 hybrid_clock.cc:648] HybridClock initialized: now 1786303272735630 us; error 30 us; skew 500 ppm
I20260809 19:21:12.736886 16320 webserver.cc:460] Webserver started at http://127.9.91.126:44881/ using document root <none> and password file <none>
I20260809 19:21:12.737103 16320 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:12.737152 16320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:12.737287 16320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:12.738236 16320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "dd0031d43b1f48dbb7b17048a2eb0a18"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "738847f0f5556f9acfbb3247e5d7a16f"
server_key_iv: "2d3e3c9437e6b00adb60be7e2f9203c7"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.738546 16320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "dd0031d43b1f48dbb7b17048a2eb0a18"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "738847f0f5556f9acfbb3247e5d7a16f"
server_key_iv: "2d3e3c9437e6b00adb60be7e2f9203c7"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.739852 16320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:21:12.740567 16342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.740720 16320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:12.740810 16320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "dd0031d43b1f48dbb7b17048a2eb0a18"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "738847f0f5556f9acfbb3247e5d7a16f"
server_key_iv: "2d3e3c9437e6b00adb60be7e2f9203c7"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.740876 16320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:12.744082 16320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:12.744314 16320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:12.744432 16320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:12.748193 16320 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:41253
I20260809 19:21:12.748229 16416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:41253 every 8 connection(s)
I20260809 19:21:12.748497 16320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:12.749076 16418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:12.750738 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 16320
I20260809 19:21:12.750828 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:12.751017 9581 external_mini_cluster.cc:1428] Setting key 59a26ddadf7f45b0e591186dcffd8b45
I20260809 19:21:12.751960 16418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18: Bootstrap starting.
I20260809 19:21:12.752666 16418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:12.753007 16418 log.cc:824] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:12.753515 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.751463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59752 (local address 127.9.91.126:41253)
0809 19:21:12.751647 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.751656 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.751688 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:21:12.752207 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.752216 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.752272 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.752440 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:12.752969 (+ 529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:12.752973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:12.752973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:12.752978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:12.753036 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:12.753050 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:12.753135 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.753201 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.753357 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260809 19:21:12.753746 16418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18: No bootstrap required, opened a new log
I20260809 19:21:12.755499 16418 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41253 } }
I20260809 19:21:12.755627 16418 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:12.755663 16418 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0031d43b1f48dbb7b17048a2eb0a18, State: Initialized, Role: FOLLOWER
I20260809 19:21:12.755777 16418 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41253 } }
I20260809 19:21:12.755849 16418 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:12.755889 16418 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:12.755939 16418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:12.756755 16418 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41253 } }
I20260809 19:21:12.756855 16418 leader_election.cc:302] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd0031d43b1f48dbb7b17048a2eb0a18; no voters:
I20260809 19:21:12.757010 16418 leader_election.cc:288] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:12.757041 16426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:12.757167 16426 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [term 1 LEADER]: Becoming Leader. State: Replica: dd0031d43b1f48dbb7b17048a2eb0a18, State: Running, Role: LEADER
I20260809 19:21:12.757273 16418 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:12.757313 16426 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41253 } }
I20260809 19:21:12.757670 16427 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41253 } } }
I20260809 19:21:12.757680 16431 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd0031d43b1f48dbb7b17048a2eb0a18. Latest consensus state: current_term: 1 leader_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0031d43b1f48dbb7b17048a2eb0a18" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41253 } } }
I20260809 19:21:12.757879 16431 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:12.757861 16427 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:12.758150 16438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:12.758586 16438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:12.760040 16438 catalog_manager.cc:1347] Generated new cluster ID: a90ba30defe74b1a985971c464a45294
I20260809 19:21:12.760085 16438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:12.787211 16438 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:12.787364 16438 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:12.793941 16438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18: Generated new TSK 0
I20260809 19:21:12.794094 16438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:12.802743 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41253
--builtin_ntp_servers=127.9.91.84:38471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:21:12.902721 16453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:12.902894 16453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:12.902943 16453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:12.904047 16453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:12.904083 16453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:12.904127 16453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:12.905447 16453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:12.905660 16453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:12.905915 16453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:12.910432 16461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:12.910478 16463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.910475 16453 server_base.cc:1034] running on GCE node
W20260809 19:21:12.910441 16460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:12.910837 16453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:12.911091 16453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:12.912246 16453 hybrid_clock.cc:648] HybridClock initialized: now 1786303272912238 us; error 28 us; skew 500 ppm
I20260809 19:21:12.913401 16453 webserver.cc:460] Webserver started at http://127.9.91.65:46875/ using document root <none> and password file <none>
I20260809 19:21:12.913614 16453 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:12.913658 16453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:12.913780 16453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:12.914728 16453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "e5389cd38ac0450f9e95d8942de05e31"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "21d7425971a0be177c4cf324aacc1b39"
server_key_iv: "df882db75194ffab47a8fed030b580cc"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.915066 16453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "e5389cd38ac0450f9e95d8942de05e31"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "21d7425971a0be177c4cf324aacc1b39"
server_key_iv: "df882db75194ffab47a8fed030b580cc"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.916227 16453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:12.916853 16472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.916981 16453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:12.917060 16453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "e5389cd38ac0450f9e95d8942de05e31"
format_stamp: "Formatted at 2026-08-09 19:21:12 on dist-test-slave-1vmg"
server_key: "21d7425971a0be177c4cf324aacc1b39"
server_key_iv: "df882db75194ffab47a8fed030b580cc"
server_key_version: "encryptionkey@0"
I20260809 19:21:12.917125 16453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:12.920241 16453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:12.920463 16453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:12.920578 16453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:12.920760 16453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:12.921021 16453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:12.921053 16453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.921078 16453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:12.921111 16453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:12.926373 16453 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:37061
I20260809 19:21:12.926425 16635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:37061 every 8 connection(s)
I20260809 19:21:12.926679 16453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:12.928349 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 16453
I20260809 19:21:12.928454 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:12.928653 9581 external_mini_cluster.cc:1428] Setting key 0bfd68735b8a943d5666d90e80e63113
I20260809 19:21:12.929464 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.927322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:37441 (local address 127.9.91.126:41253)
0809 19:21:12.927397 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:12.927400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:12.927568 (+ 168us) server_negotiation.cc:407] Connection header received
0809 19:21:12.927932 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:12.927934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:12.927940 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:12.928004 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:12.929209 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:12.929210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:12.929210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:12.929215 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:12.929233 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:12.929237 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:12.929298 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:12.929379 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:21:12.929395 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 19:21:12.929579 16639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:12.927300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:41253 (local address 127.9.91.65:37441)
0809 19:21:12.927468 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:12.927496 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:12.927859 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:12.928046 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:12.928049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:12.928056 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:12.929094 (+ 1038us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:12.929094 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:12.929121 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:12.929147 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:12.929302 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:12.929306 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:12.929309 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:12.929377 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:21:12.929422 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260809 19:21:12.929646 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41253
--builtin_ntp_servers=127.9.91.84:38471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 19:21:12.930075 16636 heartbeater.cc:347] Connected to a master server at 127.9.91.126:41253
I20260809 19:21:12.930176 16636 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:12.930331 16636 heartbeater.cc:511] Master 127.9.91.126:41253 requested a full tablet report, sending...
I20260809 19:21:12.930848 16365 ts_manager.cc:194] Registered new tserver with Master: e5389cd38ac0450f9e95d8942de05e31 (127.9.91.65:37061)
W20260809 19:21:13.010392 16642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:13.010592 16642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:13.010615 16642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:13.011757 16642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:13.011801 16642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:13.011827 16642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:13.013098 16642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:13.013278 16642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:13.013518 16642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:13.018244 16653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.018268 16651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.018275 16647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:13.018359 16642 server_base.cc:1034] running on GCE node
I20260809 19:21:13.018667 16642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:13.018884 16642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:13.020053 16642 hybrid_clock.cc:648] HybridClock initialized: now 1786303273020035 us; error 36 us; skew 500 ppm
I20260809 19:21:13.021278 16642 webserver.cc:460] Webserver started at http://127.9.91.66:39099/ using document root <none> and password file <none>
I20260809 19:21:13.021499 16642 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:13.021550 16642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:13.021687 16642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:13.022684 16642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "3249f00c97714c9397f7d8171cd02d03"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "257387cc96d7ade0c569597000ba5247"
server_key_iv: "55bbd5d2b8079a29d33c0d7e786445f6"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.023057 16642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "3249f00c97714c9397f7d8171cd02d03"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "257387cc96d7ade0c569597000ba5247"
server_key_iv: "55bbd5d2b8079a29d33c0d7e786445f6"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.024444 16642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:13.025131 16664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.025281 16642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:13.025367 16642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "3249f00c97714c9397f7d8171cd02d03"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "257387cc96d7ade0c569597000ba5247"
server_key_iv: "55bbd5d2b8079a29d33c0d7e786445f6"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.025437 16642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:13.028715 16642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:13.028949 16642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:13.029079 16642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:13.029274 16642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:13.029574 16642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:13.029613 16642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.029641 16642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:13.029668 16642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.035656 16642 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:36861
I20260809 19:21:13.035724 16841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:36861 every 8 connection(s)
I20260809 19:21:13.035990 16642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:13.038365 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.036556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:33425 (local address 127.9.91.126:41253)
0809 19:21:13.036604 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.036607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.036822 (+ 215us) server_negotiation.cc:407] Connection header received
0809 19:21:13.037207 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.037208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.037213 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.037256 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.038193 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.038194 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.038195 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.038198 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.038212 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.038216 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.038250 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.038302 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.038313 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:21:13.038539 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.036541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:41253 (local address 127.9.91.66:33425)
0809 19:21:13.036722 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.036751 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.037131 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.037309 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.037311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.037319 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:13.038066 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:13.038066 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:13.038090 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:13.038115 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:13.038250 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:13.038252 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:13.038254 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:13.038303 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:21:13.038347 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260809 19:21:13.039000 16843 heartbeater.cc:347] Connected to a master server at 127.9.91.126:41253
I20260809 19:21:13.039085 16843 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:13.039229 16843 heartbeater.cc:511] Master 127.9.91.126:41253 requested a full tablet report, sending...
I20260809 19:21:13.039554 16365 ts_manager.cc:194] Registered new tserver with Master: 3249f00c97714c9397f7d8171cd02d03 (127.9.91.66:36861)
I20260809 19:21:13.044883 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 16642
I20260809 19:21:13.044970 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:13.045132 9581 external_mini_cluster.cc:1428] Setting key 0f59ade6bcfd87caef43735a2a90786d
I20260809 19:21:13.045917 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41253
--builtin_ntp_servers=127.9.91.84:38471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 19:21:13.124850 16846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:13.125020 16846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:13.125041 16846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:13.126144 16846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:13.126183 16846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:13.126211 16846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:13.127517 16846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:13.127751 16846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:13.128019 16846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:13.132489 16856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.132495 16859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.132495 16854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:13.132830 16846 server_base.cc:1034] running on GCE node
I20260809 19:21:13.133029 16846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:13.133241 16846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:13.134369 16846 hybrid_clock.cc:648] HybridClock initialized: now 1786303273134396 us; error 64 us; skew 500 ppm
I20260809 19:21:13.135495 16846 webserver.cc:460] Webserver started at http://127.9.91.67:39531/ using document root <none> and password file <none>
I20260809 19:21:13.135691 16846 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:13.135738 16846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:13.135860 16846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:13.136787 16846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "2a87744a5cec495393d18669b9069b0b"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "ae054a8d2c91790f301500a3f316c4a9"
server_key_iv: "3aa9b34384896b42d45238cabf26bad9"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.137104 16846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "2a87744a5cec495393d18669b9069b0b"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "ae054a8d2c91790f301500a3f316c4a9"
server_key_iv: "3aa9b34384896b42d45238cabf26bad9"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.138437 16846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:13.139299 16866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.139438 16846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.139518 16846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "2a87744a5cec495393d18669b9069b0b"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "ae054a8d2c91790f301500a3f316c4a9"
server_key_iv: "3aa9b34384896b42d45238cabf26bad9"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.139583 16846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:13.142678 16846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:13.142900 16846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:13.143045 16846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:13.143239 16846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:13.143505 16846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:13.143537 16846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.143563 16846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:13.143587 16846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.149080 16846 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:39075
I20260809 19:21:13.149133 17035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:39075 every 8 connection(s)
I20260809 19:21:13.149399 16846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:13.150532 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 16846
I20260809 19:21:13.150645 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:13.150846 9581 external_mini_cluster.cc:1428] Setting key 842f60a706bb53251a3f2a89d93cee83
I20260809 19:21:13.152061 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.149960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:32883 (local address 127.9.91.126:41253)
0809 19:21:13.150004 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.150005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.150205 (+ 200us) server_negotiation.cc:407] Connection header received
0809 19:21:13.150727 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.150729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.150735 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.150784 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.151885 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.151886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.151886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.151890 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.151904 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.151907 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.151945 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.151996 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.152012 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:21:13.152232 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.149944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:41253 (local address 127.9.91.67:32883)
0809 19:21:13.150100 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.150128 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.150605 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.150850 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.150854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.150864 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:13.151763 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:13.151764 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:13.151790 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:13.151815 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:13.151952 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:13.151955 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:13.151957 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:13.152001 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:21:13.152054 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260809 19:21:13.152661 17036 heartbeater.cc:347] Connected to a master server at 127.9.91.126:41253
I20260809 19:21:13.152735 17036 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:13.152865 17036 heartbeater.cc:511] Master 127.9.91.126:41253 requested a full tablet report, sending...
I20260809 19:21:13.153151 16365 ts_manager.cc:194] Registered new tserver with Master: 2a87744a5cec495393d18669b9069b0b (127.9.91.67:39075)
I20260809 19:21:13.162319 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:21:13.164337 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.163557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.9.91.126:41253)
0809 19:21:13.163604 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.163606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.163697 (+ 91us) server_negotiation.cc:407] Connection header received
0809 19:21:13.163737 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.163738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.163742 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.163784 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.164188 (+ 404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.164188 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.164189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.164192 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.164202 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.164206 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.164244 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.164271 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.164281 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:21:13.165576 16365 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:13.167052 16365 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:13.170228 17045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.168868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:37061 (local address 127.0.0.1:35494)
0809 19:21:13.168967 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.169002 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.169050 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.169324 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.169331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.169337 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:13.169791 (+ 454us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:13.169791 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:13.169805 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:13.169824 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:13.170085 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:13.170091 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:13.170094 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:13.170156 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:21:13.170168 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":45,"threads_started":1}
I20260809 19:21:13.170266 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.168968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.9.91.65:37061)
0809 19:21:13.169154 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.169158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.169170 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:21:13.169198 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.169200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.169210 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.169535 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.169908 (+ 373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.169915 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.169916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.169920 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.169951 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.169970 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.170060 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.170171 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.170187 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260809 19:21:13.170804 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.169141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.9.91.67:39075)
0809 19:21:13.169298 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.169302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.169309 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:13.169326 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.169328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.169335 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.169436 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.170516 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.170523 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.170524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.170528 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.170561 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.170583 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.170646 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.170719 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.170733 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260809 19:21:13.170801 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.169060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:39075 (local address 127.0.0.1:54658)
0809 19:21:13.169175 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.169183 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.169216 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.169439 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.169439 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.169442 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:13.170412 (+ 970us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:13.170413 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:13.170419 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:13.170431 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:13.170653 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:13.170654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:13.170655 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:21:13.170713 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:21:13.170728 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":78,"threads_started":1}
I20260809 19:21:13.170804 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.169229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:36861 (local address 127.0.0.1:55028)
0809 19:21:13.169323 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.169331 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.169368 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.169859 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.169860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.169862 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:13.170428 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:13.170428 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:13.170432 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:13.170440 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:13.170672 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:13.170673 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:13.170673 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 19:21:13.170713 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 19:21:13.170732 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260809 19:21:13.171072 16545 tablet_service.cc:1467] Processing CreateTablet for tablet 370a4e5b748047d0a03683cce123b7ae (DEFAULT_TABLE table=test-table [id=35e929c0d2294adeac3844a110e960eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:13.171381 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.169429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55028 (local address 127.9.91.66:36861)
0809 19:21:13.169629 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.169633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.169643 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:13.169668 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.169671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.169679 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.169762 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.170519 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.170529 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.170529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.170533 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.170563 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.170584 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.170647 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.170836 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.171296 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":87,"threads_started":1}
I20260809 19:21:13.171396 16545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370a4e5b748047d0a03683cce123b7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:13.171420 16945 tablet_service.cc:1467] Processing CreateTablet for tablet 370a4e5b748047d0a03683cce123b7ae (DEFAULT_TABLE table=test-table [id=35e929c0d2294adeac3844a110e960eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:13.171698 16945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370a4e5b748047d0a03683cce123b7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:13.173892 17051 tablet_bootstrap.cc:492] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Bootstrap starting.
I20260809 19:21:13.173898 17052 tablet_bootstrap.cc:492] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31: Bootstrap starting.
I20260809 19:21:13.174536 17051 tablet_bootstrap.cc:654] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:13.174604 17052 tablet_bootstrap.cc:654] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:13.174536 16751 tablet_service.cc:1467] Processing CreateTablet for tablet 370a4e5b748047d0a03683cce123b7ae (DEFAULT_TABLE table=test-table [id=35e929c0d2294adeac3844a110e960eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:13.174866 16751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370a4e5b748047d0a03683cce123b7ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:13.174865 17051 log.cc:824] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:13.174881 17052 log.cc:824] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:13.175598 17052 tablet_bootstrap.cc:492] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31: No bootstrap required, opened a new log
I20260809 19:21:13.175598 17051 tablet_bootstrap.cc:492] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: No bootstrap required, opened a new log
I20260809 19:21:13.175668 17052 ts_tablet_manager.cc:1397] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:13.175668 17051 ts_tablet_manager.cc:1397] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:21:13.177423 17057 tablet_bootstrap.cc:492] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Bootstrap starting.
I20260809 19:21:13.177965 17057 tablet_bootstrap.cc:654] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:13.177969 17052 raft_consensus.cc:348] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.178092 17052 raft_consensus.cc:374] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:13.178118 17052 raft_consensus.cc:729] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5389cd38ac0450f9e95d8942de05e31, State: Initialized, Role: FOLLOWER
I20260809 19:21:13.178207 17052 consensus_queue.cc:260] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.178275 17057 log.cc:824] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:13.178386 17052 ts_tablet_manager.cc:1428] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:21:13.178468 16636 heartbeater.cc:503] Master 127.9.91.126:41253 was elected leader, sending a full tablet report...
I20260809 19:21:13.179349 17057 tablet_bootstrap.cc:492] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: No bootstrap required, opened a new log
I20260809 19:21:13.179525 17057 ts_tablet_manager.cc:1397] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:13.179771 17051 raft_consensus.cc:348] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.179885 17051 raft_consensus.cc:374] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:13.179904 17051 raft_consensus.cc:729] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a87744a5cec495393d18669b9069b0b, State: Initialized, Role: FOLLOWER
I20260809 19:21:13.179991 17051 consensus_queue.cc:260] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.180220 17051 ts_tablet_manager.cc:1428] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260809 19:21:13.180256 17036 heartbeater.cc:503] Master 127.9.91.126:41253 was elected leader, sending a full tablet report...
I20260809 19:21:13.181690 17057 raft_consensus.cc:348] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.181805 17057 raft_consensus.cc:374] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:13.181835 17057 raft_consensus.cc:729] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3249f00c97714c9397f7d8171cd02d03, State: Initialized, Role: FOLLOWER
I20260809 19:21:13.181938 17057 consensus_queue.cc:260] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.182145 17057 ts_tablet_manager.cc:1428] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:21:13.182196 16843 heartbeater.cc:503] Master 127.9.91.126:41253 was elected leader, sending a full tablet report...
I20260809 19:21:13.284399 17060 raft_consensus.cc:482] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:13.284530 17060 raft_consensus.cc:504] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.284897 17060 leader_election.cc:288] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5389cd38ac0450f9e95d8942de05e31 (127.9.91.65:37061), 3249f00c97714c9397f7d8171cd02d03 (127.9.91.66:36861)
I20260809 19:21:13.286069 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.285123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:59181 (local address 127.9.91.65:37061)
0809 19:21:13.285188 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.285190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.285200 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:13.285229 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.285230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.285236 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.285292 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.285867 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.285869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.285869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.285873 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.285891 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.285896 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.285955 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.285987 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.286001 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:21:13.286073 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.285053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:37061 (local address 127.9.91.67:59181)
0809 19:21:13.285114 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.285126 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.285174 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.285351 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.285352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.285354 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:13.285781 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:13.285781 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:13.285791 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:13.285804 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:13.285949 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:13.285950 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:13.285951 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:21:13.285985 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 19:21:13.285998 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 19:21:13.286264 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.285170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:36861 (local address 127.9.91.67:33891)
0809 19:21:13.285302 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.285312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.285358 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.285472 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.285473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.285475 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:13.285937 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:13.285938 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:13.285946 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:13.285958 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:13.286130 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:13.286130 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:13.286131 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:21:13.286183 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:21:13.286195 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260809 19:21:13.286264 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.285263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:33891 (local address 127.9.91.66:36861)
0809 19:21:13.285308 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.285309 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.285351 (+ 42us) server_negotiation.cc:407] Connection header received
0809 19:21:13.285415 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.285416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.285420 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.285464 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.286040 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.286042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.286042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.286047 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.286065 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.286070 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.286126 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.286184 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.286199 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 19:21:13.286489 16844 tablet.cc:2369] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:21:13.286742 16780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370a4e5b748047d0a03683cce123b7ae" candidate_uuid: "2a87744a5cec495393d18669b9069b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3249f00c97714c9397f7d8171cd02d03" is_pre_election: true
I20260809 19:21:13.286756 16568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370a4e5b748047d0a03683cce123b7ae" candidate_uuid: "2a87744a5cec495393d18669b9069b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5389cd38ac0450f9e95d8942de05e31" is_pre_election: true
I20260809 19:21:13.286860 16780 raft_consensus.cc:2454] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a87744a5cec495393d18669b9069b0b in term 0.
I20260809 19:21:13.286893 16568 raft_consensus.cc:2454] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a87744a5cec495393d18669b9069b0b in term 0.
I20260809 19:21:13.287130 16873 leader_election.cc:302] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a87744a5cec495393d18669b9069b0b, 3249f00c97714c9397f7d8171cd02d03; no voters:
I20260809 19:21:13.287245 17060 raft_consensus.cc:2781] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:13.287299 17060 raft_consensus.cc:482] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:13.287328 17060 raft_consensus.cc:3037] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:13.288004 17060 raft_consensus.cc:504] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.288105 17060 leader_election.cc:288] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [CANDIDATE]: Term 1 election: Requested vote from peers e5389cd38ac0450f9e95d8942de05e31 (127.9.91.65:37061), 3249f00c97714c9397f7d8171cd02d03 (127.9.91.66:36861)
I20260809 19:21:13.288290 16780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370a4e5b748047d0a03683cce123b7ae" candidate_uuid: "2a87744a5cec495393d18669b9069b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3249f00c97714c9397f7d8171cd02d03"
I20260809 19:21:13.288306 16568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370a4e5b748047d0a03683cce123b7ae" candidate_uuid: "2a87744a5cec495393d18669b9069b0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5389cd38ac0450f9e95d8942de05e31"
I20260809 19:21:13.288363 16780 raft_consensus.cc:3037] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:13.288369 16568 raft_consensus.cc:3037] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:13.289040 16780 raft_consensus.cc:2454] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a87744a5cec495393d18669b9069b0b in term 1.
I20260809 19:21:13.289080 16568 raft_consensus.cc:2454] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a87744a5cec495393d18669b9069b0b in term 1.
I20260809 19:21:13.289207 16873 leader_election.cc:302] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a87744a5cec495393d18669b9069b0b, e5389cd38ac0450f9e95d8942de05e31; no voters:
I20260809 19:21:13.289319 17060 raft_consensus.cc:2781] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:13.289438 17060 raft_consensus.cc:686] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 1 LEADER]: Becoming Leader. State: Replica: 2a87744a5cec495393d18669b9069b0b, State: Running, Role: LEADER
I20260809 19:21:13.289512 17060 consensus_queue.cc:237] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } }
I20260809 19:21:13.290282 16364 catalog_manager.cc:5495] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a87744a5cec495393d18669b9069b0b (127.9.91.67). New cstate: current_term: 1 leader_uuid: "2a87744a5cec495393d18669b9069b0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a87744a5cec495393d18669b9069b0b" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:13.315891 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.315074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.9.91.67:39075)
0809 19:21:13.315130 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.315132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.315140 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:13.315188 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.315189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.315201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.315252 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.315692 (+ 440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.315694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.315694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.315698 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.315711 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.315714 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.315763 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.315816 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.315827 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:21:13.317337 16780 raft_consensus.cc:1261] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 1 FOLLOWER]: Refusing update from remote peer 2a87744a5cec495393d18669b9069b0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:13.317365 16568 raft_consensus.cc:1261] T 370a4e5b748047d0a03683cce123b7ae P e5389cd38ac0450f9e95d8942de05e31 [term 1 FOLLOWER]: Refusing update from remote peer 2a87744a5cec495393d18669b9069b0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:13.317636 17060 consensus_queue.cc:1035] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5389cd38ac0450f9e95d8942de05e31" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 37061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:13.317764 17064 consensus_queue.cc:1035] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3249f00c97714c9397f7d8171cd02d03" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:13.322736 16365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:13.322816 16365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:13.323496 16365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd0031d43b1f48dbb7b17048a2eb0a18: Sending DeleteTablet for 3 replicas of tablet 370a4e5b748047d0a03683cce123b7ae
I20260809 19:21:13.323833 16945 tablet_service.cc:1514] Processing DeleteTablet for tablet 370a4e5b748047d0a03683cce123b7ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:13 UTC) from {username='slave'} at 127.0.0.1:54658
I20260809 19:21:13.323858 16751 tablet_service.cc:1514] Processing DeleteTablet for tablet 370a4e5b748047d0a03683cce123b7ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:13 UTC) from {username='slave'} at 127.0.0.1:55028
I20260809 19:21:13.323863 16545 tablet_service.cc:1514] Processing DeleteTablet for tablet 370a4e5b748047d0a03683cce123b7ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:13 UTC) from {username='slave'} at 127.0.0.1:35494
I20260809 19:21:13.324043 17079 tablet_replica.cc:330] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: stopping tablet replica
I20260809 19:21:13.324127 17079 raft_consensus.cc:2229] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:13.324210 17079 raft_consensus.cc:2258] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:13.324505 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 16453
I20260809 19:21:13.324644 17079 ts_tablet_manager.cc:1905] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:13.326325 17079 ts_tablet_manager.cc:1918] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:13.326399 17079 log.cc:1196] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/370a4e5b748047d0a03683cce123b7ae
I20260809 19:21:13.326690 17079 ts_tablet_manager.cc:1939] T 370a4e5b748047d0a03683cce123b7ae P 3249f00c97714c9397f7d8171cd02d03: Deleting consensus metadata
I20260809 19:21:13.327152 16347 catalog_manager.cc:4841] TS 3249f00c97714c9397f7d8171cd02d03 (127.9.91.66:36861): tablet 370a4e5b748047d0a03683cce123b7ae (table test-table [id=35e929c0d2294adeac3844a110e960eb]) successfully deleted
I20260809 19:21:13.335100 17080 tablet_replica.cc:330] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: stopping tablet replica
I20260809 19:21:13.335212 17080 raft_consensus.cc:2229] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:13.335335 17080 raft_consensus.cc:2258] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 19:21:13.335568 16347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:37061 (error 108)
I20260809 19:21:13.335799 17080 ts_tablet_manager.cc:1905] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:13.336010 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.335893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:37061 (local address 127.0.0.1:35504)
0809 19:21:13.335941 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.335980 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:37061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:21:13.336126 16347 catalog_manager.cc:4584] TS e5389cd38ac0450f9e95d8942de05e31 (127.9.91.65:37061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 370a4e5b748047d0a03683cce123b7ae: Network error: Client connection negotiation failed: client connection to 127.9.91.65:37061: connect: Connection refused (error 111)
I20260809 19:21:13.336562 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 16642
I20260809 19:21:13.338502 17080 ts_tablet_manager.cc:1918] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:13.338617 17080 log.cc:1196] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/370a4e5b748047d0a03683cce123b7ae
I20260809 19:21:13.338878 17080 ts_tablet_manager.cc:1939] T 370a4e5b748047d0a03683cce123b7ae P 2a87744a5cec495393d18669b9069b0b: Deleting consensus metadata
I20260809 19:21:13.340433 16347 catalog_manager.cc:4841] TS 2a87744a5cec495393d18669b9069b0b (127.9.91.67:39075): tablet 370a4e5b748047d0a03683cce123b7ae (table test-table [id=35e929c0d2294adeac3844a110e960eb]) successfully deleted
I20260809 19:21:13.344106 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 16846
I20260809 19:21:13.351191 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 16320
2026-08-09T19:21:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (733 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 19:21:13.373401 9581 mini_oidc.cc:152] Starting JWKS server
I20260809 19:21:13.374500 9581 webserver.cc:460] Webserver started at https://127.0.0.1:42709/ using document root <none> and password file <none>
I20260809 19:21:13.374588 9581 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:21:13.374749 9581 webserver.cc:460] Webserver started at http://127.0.0.1:33883/ using document root <none> and password file <none>
2026-08-09T19:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:13Z Disabled control of system clock
I20260809 19:21:13.382432 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37289
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:37775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:37289
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42709/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/testchainca.cert with env {}
W20260809 19:21:13.459759 17094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:13.459933 17094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:13.459955 17094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:13.461040 17094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:13.461081 17094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:13.461092 17094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:13.461103 17094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:13.461115 17094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:21:13.461131 17094 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:21:13.461141 17094 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42709/jwks.json
I20260809 19:21:13.462574 17094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:37775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:37289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42709/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:13.462751 17094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:13.462898 17094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:13.467511 17109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.467494 17106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.467510 17107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:13.467657 17094 server_base.cc:1034] running on GCE node
I20260809 19:21:13.467872 17094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:13.468149 17094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:13.469286 17094 hybrid_clock.cc:648] HybridClock initialized: now 1786303273469274 us; error 30 us; skew 500 ppm
I20260809 19:21:13.470345 17094 webserver.cc:460] Webserver started at http://127.9.91.126:35335/ using document root <none> and password file <none>
I20260809 19:21:13.470541 17094 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:13.470589 17094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:13.470708 17094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:13.471697 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "21d95f828d594079aae7ac504aeaa257"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "2c46f6b439acc6c0517a055310d9843a"
server_key_iv: "25a0e587e98d5cd6d9be802a3b003281"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.472014 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "21d95f828d594079aae7ac504aeaa257"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "2c46f6b439acc6c0517a055310d9843a"
server_key_iv: "25a0e587e98d5cd6d9be802a3b003281"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.473168 17094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:21:13.473806 17117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.473940 17094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:13.474020 17094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "21d95f828d594079aae7ac504aeaa257"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "2c46f6b439acc6c0517a055310d9843a"
server_key_iv: "25a0e587e98d5cd6d9be802a3b003281"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.474088 17094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:13.488602 17094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:13.488842 17094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:13.488962 17094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:13.492646 17094 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:37289
I20260809 19:21:13.492678 17189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:37289 every 8 connection(s)
I20260809 19:21:13.492955 17094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:13.493808 17190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:13.495949 17190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257: Bootstrap starting.
I20260809 19:21:13.496276 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 17094
I20260809 19:21:13.496389 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:13.496585 9581 external_mini_cluster.cc:1428] Setting key 066cdc9e1386ecea7b502f793af3ae10
I20260809 19:21:13.496709 17190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:13.497036 17190 log.cc:824] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:13.497891 17190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257: No bootstrap required, opened a new log
I20260809 19:21:13.499928 17190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d95f828d594079aae7ac504aeaa257" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37289 } }
I20260809 19:21:13.500087 17190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:13.500120 17190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21d95f828d594079aae7ac504aeaa257, State: Initialized, Role: FOLLOWER
I20260809 19:21:13.500241 17190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d95f828d594079aae7ac504aeaa257" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37289 } }
I20260809 19:21:13.500314 17190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:13.500345 17190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:13.500392 17190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:13.500950 17190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d95f828d594079aae7ac504aeaa257" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37289 } }
I20260809 19:21:13.501060 17190 leader_election.cc:302] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21d95f828d594079aae7ac504aeaa257; no voters:
I20260809 19:21:13.501215 17190 leader_election.cc:288] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:13.501325 17198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:13.501466 17198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [term 1 LEADER]: Becoming Leader. State: Replica: 21d95f828d594079aae7ac504aeaa257, State: Running, Role: LEADER
I20260809 19:21:13.501544 17190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:13.501556 17198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d95f828d594079aae7ac504aeaa257" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37289 } }
I20260809 19:21:13.501853 17198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21d95f828d594079aae7ac504aeaa257. Latest consensus state: current_term: 1 leader_uuid: "21d95f828d594079aae7ac504aeaa257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d95f828d594079aae7ac504aeaa257" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37289 } } }
I20260809 19:21:13.501940 17198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:13.501866 17199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21d95f828d594079aae7ac504aeaa257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21d95f828d594079aae7ac504aeaa257" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37289 } } }
I20260809 19:21:13.502084 17199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:13.502413 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.497044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.9.91.126:37289)
0809 19:21:13.497263 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.497272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.497303 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:21:13.497795 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.497802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.497861 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.497993 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.498237 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.499133 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.500223 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.500697 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.500784 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.501528 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.501532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.501532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.501728 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.501775 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.501790 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.501847 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.502137 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.502226 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260809 19:21:13.503330 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 19:21:13.503484 17218 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:21:13.503530 17218 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:21:13.503803 17206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:13.503974 17206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:13.505353 17206 catalog_manager.cc:1347] Generated new cluster ID: 9f1193e24ecd41988d907f9db6e6a33f
I20260809 19:21:13.505409 17206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:13.513091 17206 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:13.513623 17206 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:13.515722 17206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21d95f828d594079aae7ac504aeaa257: Generated new TSK 0
I20260809 19:21:13.515875 17206 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 19:21:13.540907 17231 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 19:21:13.541003 17231 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 19:21:13.541262 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.526015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.9.91.126:37289)
0809 19:21:13.526058 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.526060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.526184 (+ 124us) server_negotiation.cc:407] Connection header received
0809 19:21:13.526247 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.526248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.526259 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.526338 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:21:13.526452 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.527215 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.527963 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.528265 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.528330 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.528479 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:21:13.541063 (+ 12584us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42709/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 19:21:13.541200 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60308: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42709/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 19:21:13.541326 17196 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60308: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42709/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 19:21:13.541599 17230 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:13.525943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37289 (local address 127.0.0.1:60308)
0809 19:21:13.526116 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.526125 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.526178 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.526339 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.526340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.526341 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:21:13.526399 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:13.526400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.527307 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.527308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:13.527890 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:13.527892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.528327 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.528327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:13.528366 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.528384 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:21:13.541168 (+ 12784us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42709/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 19:21:13.541251 (+ 83us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.91.126:37289: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42709/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.91.126:37289: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42709/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 19:21:13.541982 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 17094
2026-08-09T19:21:13Z chronyd exiting
I20260809 19:21:13.589107 9581 test_util.cc:175] -----------------------------------------------
I20260809 19:21:13.589165 9581 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786303248613589-9581-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 19:21:13.589951 9581 mini_oidc.cc:152] Starting JWKS server
I20260809 19:21:13.590576 9581 webserver.cc:460] Webserver started at https://127.0.0.1:40671/ using document root <none> and password file <none>
I20260809 19:21:13.590615 9581 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:21:13.590749 9581 webserver.cc:460] Webserver started at http://127.0.0.1:37131/ using document root <none> and password file <none>
2026-08-09T19:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:13Z Disabled control of system clock
I20260809 19:21:13.598842 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35367
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:37797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:35367
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/testchainca.cert with env {}
W20260809 19:21:13.676390 17238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:13.676573 17238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:13.676602 17238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:13.677706 17238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:13.677744 17238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:13.677760 17238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:13.677776 17238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:13.677791 17238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:21:13.677809 17238 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:21:13.677826 17238 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40671/jwks.json
I20260809 19:21:13.679286 17238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:37797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:35367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:13.679512 17238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:13.679692 17238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:13.684185 17248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.684163 17250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.684168 17246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:13.684229 17238 server_base.cc:1034] running on GCE node
I20260809 19:21:13.684578 17238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:13.684837 17238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:13.685968 17238 hybrid_clock.cc:648] HybridClock initialized: now 1786303273685960 us; error 26 us; skew 500 ppm
I20260809 19:21:13.687065 17238 webserver.cc:460] Webserver started at http://127.9.91.126:35475/ using document root <none> and password file <none>
I20260809 19:21:13.687258 17238 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:13.687300 17238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:13.687418 17238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:13.688424 17238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "9548b34afc2e55e5ef0b68341df91a00"
server_key_iv: "8b1c905c0fecb9c362db50a6f247534d"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.688741 17238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "9548b34afc2e55e5ef0b68341df91a00"
server_key_iv: "8b1c905c0fecb9c362db50a6f247534d"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.690115 17238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:13.690829 17260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.691031 17238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:13.691111 17238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "9548b34afc2e55e5ef0b68341df91a00"
server_key_iv: "8b1c905c0fecb9c362db50a6f247534d"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.691179 17238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:13.712981 17238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:13.713222 17238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:13.713337 17238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:13.716892 17238 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:35367
I20260809 19:21:13.716925 17340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:35367 every 8 connection(s)
I20260809 19:21:13.717203 17238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:13.718051 17341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:13.720116 17341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1: Bootstrap starting.
I20260809 19:21:13.720681 17341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:13.720917 17341 log.cc:824] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:13.721486 17341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1: No bootstrap required, opened a new log
I20260809 19:21:13.722551 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 17238
I20260809 19:21:13.722656 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:13.722826 9581 external_mini_cluster.cc:1428] Setting key bf629960d6047fcfc521421e37d3302a
I20260809 19:21:13.723112 17341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35367 } }
I20260809 19:21:13.723274 17341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:13.723318 17341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ffa8ee445564e8d9ab3aad494ea6ae1, State: Initialized, Role: FOLLOWER
I20260809 19:21:13.723444 17341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35367 } }
I20260809 19:21:13.723522 17341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:13.723552 17341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:13.723608 17341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:13.724186 17341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35367 } }
I20260809 19:21:13.724284 17341 leader_election.cc:302] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ffa8ee445564e8d9ab3aad494ea6ae1; no voters:
I20260809 19:21:13.724428 17341 leader_election.cc:288] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:13.724607 17348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:13.724678 17341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:13.724761 17348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [term 1 LEADER]: Becoming Leader. State: Replica: 3ffa8ee445564e8d9ab3aad494ea6ae1, State: Running, Role: LEADER
I20260809 19:21:13.724848 17348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35367 } }
I20260809 19:21:13.725153 17348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ffa8ee445564e8d9ab3aad494ea6ae1. Latest consensus state: current_term: 1 leader_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35367 } } }
I20260809 19:21:13.725214 17348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:13.725188 17349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ffa8ee445564e8d9ab3aad494ea6ae1" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35367 } } }
I20260809 19:21:13.725265 17349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1 [sys.catalog]: This master's current role is: LEADER
W20260809 19:21:13.726471 17371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:21:13.726518 17371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:21:13.726569 17364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:13.726742 17364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:13.728369 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.723299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.9.91.126:35367)
0809 19:21:13.723482 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.723489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.723511 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:21:13.723609 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.723616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.723673 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.723780 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.723994 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.725276 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.726377 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.726986 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.727114 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.727678 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.727681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.727682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.727867 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.727912 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.727927 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.727993 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.728084 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.728161 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260809 19:21:13.728376 17364 catalog_manager.cc:1347] Generated new cluster ID: fbaf8a12c0994cdd8fd337114726babe
I20260809 19:21:13.728451 17364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:13.729182 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:21:13.736935 17364 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:13.737381 17364 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:13.743793 17364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ffa8ee445564e8d9ab3aad494ea6ae1: Generated new TSK 0
I20260809 19:21:13.743938 17364 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 19:21:13.771097 17385 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 19:21:13.771164 17385 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 19:21:13.771427 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.749984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.9.91.126:35367)
0809 19:21:13.750018 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.750019 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.750140 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:21:13.750198 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.750199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.750210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.750273 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:21:13.750386 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.750845 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.751618 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.751824 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.751877 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.752030 (+ 153us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:21:13.771227 (+ 19197us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 19:21:13.771368 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40758: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:21:13.771484 17347 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40758: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 19:21:13.771543 17383 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:13.749929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35367 (local address 127.0.0.1:40758)
0809 19:21:13.750076 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.750086 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.750145 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.750274 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.750274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.750276 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:21:13.750326 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:13.750328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.750957 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.750958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:13.751543 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:13.751545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.751886 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.751887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:13.751944 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.751949 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:21:13.771341 (+ 19392us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 19:21:13.771451 (+ 110us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.91.126:35367: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40671/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260809 19:21:13.771757 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 17238
2026-08-09T19:21:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 19:21:13.814608 9581 mini_oidc.cc:152] Starting JWKS server
I20260809 19:21:13.815289 9581 webserver.cc:460] Webserver started at https://127.0.0.1:40755/ using document root <none> and password file <none>
I20260809 19:21:13.815331 9581 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 19:21:13.815478 9581 webserver.cc:460] Webserver started at http://127.0.0.1:42977/ using document root <none> and password file <none>
2026-08-09T19:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:13Z Disabled control of system clock
I20260809 19:21:13.823611 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37601
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:35537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:37601
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40755/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 19:21:13.901154 17393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:13.901328 17393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:13.901351 17393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:13.902420 17393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:13.902454 17393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:13.902465 17393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:13.902477 17393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:13.902490 17393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:21:13.902501 17393 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:21:13.902511 17393 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40755/jwks.json
I20260809 19:21:13.903826 17393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:37601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40755/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:13.904011 17393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:13.904227 17393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:13.908573 17401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:13.908640 17399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:13.908622 17393 server_base.cc:1034] running on GCE node
W20260809 19:21:13.908783 17398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:13.909049 17393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:13.909307 17393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:13.910452 17393 hybrid_clock.cc:648] HybridClock initialized: now 1786303273910440 us; error 30 us; skew 500 ppm
I20260809 19:21:13.911621 17393 webserver.cc:460] Webserver started at http://127.9.91.126:34603/ using document root <none> and password file <none>
I20260809 19:21:13.911823 17393 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:13.911864 17393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:13.911983 17393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:13.913014 17393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "fa13d69000cd40559cb505f82a970e0d"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "aef52f7bf511ca2e739d5566a213127d"
server_key_iv: "c32dc25c3b423dd23c20347b01dd8238"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.913400 17393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "fa13d69000cd40559cb505f82a970e0d"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "aef52f7bf511ca2e739d5566a213127d"
server_key_iv: "c32dc25c3b423dd23c20347b01dd8238"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.914635 17393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:13.915325 17409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.915462 17393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:13.915545 17393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "fa13d69000cd40559cb505f82a970e0d"
format_stamp: "Formatted at 2026-08-09 19:21:13 on dist-test-slave-1vmg"
server_key: "aef52f7bf511ca2e739d5566a213127d"
server_key_iv: "c32dc25c3b423dd23c20347b01dd8238"
server_key_version: "encryptionkey@0"
I20260809 19:21:13.915608 17393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:13.934978 17393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:13.935210 17393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:13.935322 17393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:13.938954 17393 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:37601
I20260809 19:21:13.938968 17490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:37601 every 8 connection(s)
I20260809 19:21:13.939275 17393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:13.939894 17491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:13.942387 17491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d: Bootstrap starting.
I20260809 19:21:13.942993 17491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:13.943262 17491 log.cc:824] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:13.943876 17491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d: No bootstrap required, opened a new log
I20260809 19:21:13.945374 17491 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa13d69000cd40559cb505f82a970e0d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37601 } }
I20260809 19:21:13.945493 17491 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:13.945524 17491 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa13d69000cd40559cb505f82a970e0d, State: Initialized, Role: FOLLOWER
I20260809 19:21:13.945652 17491 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa13d69000cd40559cb505f82a970e0d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37601 } }
I20260809 19:21:13.945725 17491 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:13.945760 17491 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:13.945806 17491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:13.946285 17491 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa13d69000cd40559cb505f82a970e0d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37601 } }
I20260809 19:21:13.946370 17491 leader_election.cc:302] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa13d69000cd40559cb505f82a970e0d; no voters:
I20260809 19:21:13.946519 17491 leader_election.cc:288] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:13.946557 17497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:13.946681 17497 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [term 1 LEADER]: Becoming Leader. State: Replica: fa13d69000cd40559cb505f82a970e0d, State: Running, Role: LEADER
I20260809 19:21:13.946753 17491 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:13.946789 17497 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa13d69000cd40559cb505f82a970e0d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37601 } }
I20260809 19:21:13.947186 17500 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa13d69000cd40559cb505f82a970e0d. Latest consensus state: current_term: 1 leader_uuid: "fa13d69000cd40559cb505f82a970e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa13d69000cd40559cb505f82a970e0d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37601 } } }
I20260809 19:21:13.947273 17500 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:13.947378 17498 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa13d69000cd40559cb505f82a970e0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa13d69000cd40559cb505f82a970e0d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37601 } } }
I20260809 19:21:13.947465 17498 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:13.947474 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 17393
I20260809 19:21:13.947580 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:13.947748 9581 external_mini_cluster.cc:1428] Setting key 84df0551df3be00459b77f4c88393857
I20260809 19:21:13.947865 17507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:13.948364 17507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:13.949843 17507 catalog_manager.cc:1347] Generated new cluster ID: 623aa9e8a26f4b51927d82cf1093dc5a
I20260809 19:21:13.949894 17507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:13.953998 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.948342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.9.91.126:37601)
0809 19:21:13.948922 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.948930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.948961 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:21:13.949061 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.949066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.949113 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.949244 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:13.949633 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.950944 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.952022 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.952635 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.952733 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.953376 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:13.953379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:13.953379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:13.953562 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:13.953600 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:13.953612 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:13.953666 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:13.953744 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 19:21:13.953810 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":85,"threads_started":1}
I20260809 19:21:13.954833 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:21:13.962708 17507 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:13.963207 17507 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:13.968815 17507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa13d69000cd40559cb505f82a970e0d: Generated new TSK 0
I20260809 19:21:13.968962 17507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:13.979552 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:13.976463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.9.91.126:37601)
0809 19:21:13.976505 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:13.976507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:13.976576 (+ 69us) server_negotiation.cc:407] Connection header received
0809 19:21:13.976652 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:13.976652 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:13.976661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:13.976711 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:21:13.976815 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.977275 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.978324 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.978488 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.978528 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.978686 (+ 158us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:21:13.979381 (+ 695us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40755/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 19:21:13.979489 (+ 108us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41844: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40755/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 19:21:13.979607 17523 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41844: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40755/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 19:21:13.979671 17543 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:13.976385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37601 (local address 127.0.0.1:41844)
0809 19:21:13.976519 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:13.976529 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:13.976593 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:13.976712 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:13.976713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:13.976715 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:21:13.976765 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:13.976767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.977342 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.977342 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:13.978253 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:13.978257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:13.978549 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:13.978550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:13.978606 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:13.978612 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:21:13.979478 (+ 866us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40755/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 19:21:13.979577 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.91.126:37601: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40755/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260809 19:21:13.979964 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 17393
E20260809 19:21:13.996284 17544 webserver.cc:533] Webserver: sslize failed:
E20260809 19:21:13.996345 17544 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T19:21:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (209 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T19:21:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:14Z Disabled control of system clock
I20260809 19:21:14.032249 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40961
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:40961
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/testchainca.cert with env {}
W20260809 19:21:14.110821 17549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:14.111025 17549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:14.111049 17549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:14.112146 17549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:14.112178 17549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:14.112190 17549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:14.112200 17549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:14.112211 17549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 19:21:14.112228 17549 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 19:21:14.112239 17549 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 19:21:14.113631 17549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:40961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:14.113804 17549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:14.113951 17549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:14.118366 17563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:14.118407 17549 server_base.cc:1034] running on GCE node
W20260809 19:21:14.118383 17565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:14.118379 17562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:14.118826 17549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:14.119143 17549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:14.120285 17549 hybrid_clock.cc:648] HybridClock initialized: now 1786303274120270 us; error 34 us; skew 500 ppm
I20260809 19:21:14.121508 17549 webserver.cc:460] Webserver started at http://127.9.91.126:41299/ using document root <none> and password file <none>
I20260809 19:21:14.121718 17549 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:14.121762 17549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:14.121865 17549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:14.122864 17549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "fe3ab4b623154268a378e99c2a050e3a"
format_stamp: "Formatted at 2026-08-09 19:21:14 on dist-test-slave-1vmg"
server_key: "15b8236f9743a50fdc4af98777ddee5f"
server_key_iv: "db531f4b281e575cf7cd58e0593f3454"
server_key_version: "encryptionkey@0"
I20260809 19:21:14.123242 17549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "fe3ab4b623154268a378e99c2a050e3a"
format_stamp: "Formatted at 2026-08-09 19:21:14 on dist-test-slave-1vmg"
server_key: "15b8236f9743a50fdc4af98777ddee5f"
server_key_iv: "db531f4b281e575cf7cd58e0593f3454"
server_key_version: "encryptionkey@0"
I20260809 19:21:14.124588 17549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:14.125252 17574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:14.125392 17549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:14.125483 17549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "fe3ab4b623154268a378e99c2a050e3a"
format_stamp: "Formatted at 2026-08-09 19:21:14 on dist-test-slave-1vmg"
server_key: "15b8236f9743a50fdc4af98777ddee5f"
server_key_iv: "db531f4b281e575cf7cd58e0593f3454"
server_key_version: "encryptionkey@0"
I20260809 19:21:14.125550 17549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:14.139612 17549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:14.139896 17549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:14.140046 17549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:14.144352 17549 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:40961
I20260809 19:21:14.144382 17652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:40961 every 8 connection(s)
I20260809 19:21:14.144717 17549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:14.145658 17654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:14.146090 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 17549
I20260809 19:21:14.146194 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:14.146386 9581 external_mini_cluster.cc:1428] Setting key 3f920945bd698f25f660d3ad5df7c475
I20260809 19:21:14.148381 17654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a: Bootstrap starting.
I20260809 19:21:14.148974 17654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:14.149292 17654 log.cc:824] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:14.150038 17654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a: No bootstrap required, opened a new log
I20260809 19:21:14.151881 17660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:14.146897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.9.91.126:40961)
0809 19:21:14.147136 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:14.147143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:14.147169 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:21:14.147606 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:14.147613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:14.147674 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:14.147793 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:14.148037 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:14.148945 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:14.149926 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:14.150364 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:14.150461 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:14.151236 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:14.151240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:14.151240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:14.151409 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:14.151447 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:14.151458 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:14.151525 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:14.151620 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:21:14.151684 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260809 19:21:14.152137 17654 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3ab4b623154268a378e99c2a050e3a" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40961 } }
I20260809 19:21:14.152278 17654 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:14.152302 17654 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3ab4b623154268a378e99c2a050e3a, State: Initialized, Role: FOLLOWER
I20260809 19:21:14.152401 17654 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3ab4b623154268a378e99c2a050e3a" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40961 } }
I20260809 19:21:14.152467 17654 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:14.152503 17654 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:14.152547 17654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:14.152848 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:21:14.153108 17654 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3ab4b623154268a378e99c2a050e3a" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40961 } }
I20260809 19:21:14.153208 17654 leader_election.cc:302] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe3ab4b623154268a378e99c2a050e3a; no voters:
I20260809 19:21:14.153353 17654 leader_election.cc:288] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:14.153416 17663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:14.153573 17663 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [term 1 LEADER]: Becoming Leader. State: Replica: fe3ab4b623154268a378e99c2a050e3a, State: Running, Role: LEADER
I20260809 19:21:14.153590 17654 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:14.153661 17663 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3ab4b623154268a378e99c2a050e3a" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40961 } }
I20260809 19:21:14.153980 17663 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe3ab4b623154268a378e99c2a050e3a. Latest consensus state: current_term: 1 leader_uuid: "fe3ab4b623154268a378e99c2a050e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3ab4b623154268a378e99c2a050e3a" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40961 } } }
I20260809 19:21:14.154067 17663 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:14.154088 17664 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe3ab4b623154268a378e99c2a050e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe3ab4b623154268a378e99c2a050e3a" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40961 } } }
I20260809 19:21:14.154173 17664 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:14.154856 17671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:14.155311 17671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:14.156628 17671 catalog_manager.cc:1347] Generated new cluster ID: 108b82b9da084aef97c65ef8c4cd5180
I20260809 19:21:14.156678 17671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:14.161906 17671 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:14.162585 17671 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:14.167886 17671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe3ab4b623154268a378e99c2a050e3a: Generated new TSK 0
I20260809 19:21:14.168071 17671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:14.209298 17660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:14.175750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.9.91.126:40961)
0809 19:21:14.175805 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:14.175808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:14.175903 (+ 95us) server_negotiation.cc:407] Connection header received
0809 19:21:14.175977 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:14.175978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:14.175992 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:14.176069 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0809 19:21:14.176201 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:14.176682 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:14.177435 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:14.177656 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:14.177713 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:14.177883 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 19:21:14.209028 (+ 31145us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 19:21:14.209217 (+ 189us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33982: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 19:21:14.209389 17660 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33982: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 19:21:14.209414 17695 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:14.175692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40961 (local address 127.0.0.1:33982)
0809 19:21:14.175842 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:14.175851 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:14.175909 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:14.176081 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:14.176082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:14.176084 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 19:21:14.176145 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:14.176147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:14.176775 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:14.176776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:14.177360 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:14.177362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:14.177733 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:14.177734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:14.177795 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:14.177801 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 19:21:14.209192 (+ 31391us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 19:21:14.209316 (+ 124us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.91.126:40961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260809 19:21:14.209761 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 17549
2026-08-09T19:21:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (210 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (89 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:14 dist-test-slave-1vmg krb5kdc[17718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:14 dist-test-slave-1vmg krb5kdc[17718](info): set up 2 sockets
Aug 09 19:21:14 dist-test-slave-1vmg krb5kdc[17718](info): commencing operation
krb5kdc: starting...
W20260809 19:21:15.514528 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:15 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:15.621352 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46241
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:37545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:46241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:15.700520 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:15.700701 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:15.700722 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:15.701797 17736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:15.701836 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:15.701849 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:15.701860 17736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:15.701877 17736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:15.703303 17736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:37545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:46241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:15.703486 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:15.703647 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:15.708081 17745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:15.708089 17743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:15.708115 17747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:15.708312 17736 server_base.cc:1034] running on GCE node
I20260809 19:21:15.708514 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:15.708767 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:15.709901 17736 hybrid_clock.cc:648] HybridClock initialized: now 1786303275709888 us; error 28 us; skew 500 ppm
Aug 09 19:21:15 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303275, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:15.715140 17736 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:15.715543 17736 webserver.cc:460] Webserver started at http://127.9.91.126:35847/ using document root <none> and password file <none>
I20260809 19:21:15.715739 17736 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:15.715785 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:15.715903 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:15.716835 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "e7254684d2f644b8bd44011eac937176"
format_stamp: "Formatted at 2026-08-09 19:21:15 on dist-test-slave-1vmg"
server_key: "3c58aaa7b8df8617ffeb92d77979e4f2"
server_key_iv: "eb6f3d258d85be7db44ee37b606d8b3c"
server_key_version: "encryptionkey@0"
I20260809 19:21:15.717160 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "e7254684d2f644b8bd44011eac937176"
format_stamp: "Formatted at 2026-08-09 19:21:15 on dist-test-slave-1vmg"
server_key: "3c58aaa7b8df8617ffeb92d77979e4f2"
server_key_iv: "eb6f3d258d85be7db44ee37b606d8b3c"
server_key_version: "encryptionkey@0"
I20260809 19:21:15.718604 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:21:15.719342 17762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:15.719494 17736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:15.719576 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "e7254684d2f644b8bd44011eac937176"
format_stamp: "Formatted at 2026-08-09 19:21:15 on dist-test-slave-1vmg"
server_key: "3c58aaa7b8df8617ffeb92d77979e4f2"
server_key_iv: "eb6f3d258d85be7db44ee37b606d8b3c"
server_key_version: "encryptionkey@0"
I20260809 19:21:15.719637 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:15.726356 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:15.727952 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:15.728081 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:15.732017 17736 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:46241
I20260809 19:21:15.732061 17845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:46241 every 8 connection(s)
I20260809 19:21:15.732358 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:15.733119 17846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:15.735105 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 17736
I20260809 19:21:15.735221 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:15.735246 17846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176: Bootstrap starting.
I20260809 19:21:15.735406 9581 external_mini_cluster.cc:1428] Setting key 1672808d92f5ac3dd5c1b8fd5353ced8
I20260809 19:21:15.735874 17846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:15.736207 17846 log.cc:824] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:15.736891 17846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176: No bootstrap required, opened a new log
I20260809 19:21:15.738983 17846 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7254684d2f644b8bd44011eac937176" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46241 } }
I20260809 19:21:15.739144 17846 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:15.739174 17846 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7254684d2f644b8bd44011eac937176, State: Initialized, Role: FOLLOWER
I20260809 19:21:15.739283 17846 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7254684d2f644b8bd44011eac937176" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46241 } }
I20260809 19:21:15.739353 17846 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:15.739435 17846 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:15.739481 17846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:15.740027 17846 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7254684d2f644b8bd44011eac937176" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46241 } }
I20260809 19:21:15.740135 17846 leader_election.cc:302] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7254684d2f644b8bd44011eac937176; no voters:
I20260809 19:21:15.740286 17846 leader_election.cc:288] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:15.740327 17853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:15.740517 17846 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:15.740526 17853 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [term 1 LEADER]: Becoming Leader. State: Replica: e7254684d2f644b8bd44011eac937176, State: Running, Role: LEADER
I20260809 19:21:15.740672 17853 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7254684d2f644b8bd44011eac937176" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46241 } }
I20260809 19:21:15.741077 17858 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7254684d2f644b8bd44011eac937176. Latest consensus state: current_term: 1 leader_uuid: "e7254684d2f644b8bd44011eac937176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7254684d2f644b8bd44011eac937176" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46241 } } }
I20260809 19:21:15.741207 17858 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:15.741433 17854 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7254684d2f644b8bd44011eac937176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7254684d2f644b8bd44011eac937176" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46241 } } }
I20260809 19:21:15.741557 17854 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176 [sys.catalog]: This master's current role is: LEADER
Aug 09 19:21:15 dist-test-slave-1vmg krb5kdc[17718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
W20260809 19:21:15.742709 17876 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:21:15.742777 17876 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:21:15.742821 17860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:15.742993 17860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:15.744429 17860 catalog_manager.cc:1347] Generated new cluster ID: aa37563086b6451ea6bffd3f96893bb0
I20260809 19:21:15.744482 17860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:15.745925 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:15.735888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.9.91.126:46241)
0809 19:21:15.736053 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:15.736062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:15.736089 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:21:15.736539 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:15.736548 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:15.736609 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:15.736740 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:15.737507 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:15.738346 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:15.739399 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:15.739873 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:15.739950 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:15.742669 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:15.742673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:15.742674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:15.742685 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:15.744609 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:15.744896 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:15.744899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:15.744899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:15.744985 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:15.745154 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:15.745154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:15.745155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:15.745365 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:15.745483 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:15.745600 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 19:21:15.745725 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260809 19:21:15.757019 17860 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:15.757462 17860 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:15.762496 17860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7254684d2f644b8bd44011eac937176: Generated new TSK 0
I20260809 19:21:15.762645 17860 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:21:15.791538 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46241
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:37545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:15.871834 17891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:15.872020 17891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:15.872042 17891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:15.873114 17891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:15.873147 17891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:15.873170 17891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:15.874601 17891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:37545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:15.874787 17891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:15.874974 17891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:15.879573 17902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:15.879571 17900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:15.879734 17899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:15.879668 17891 server_base.cc:1034] running on GCE node
I20260809 19:21:15.879977 17891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:15.880189 17891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:15.881330 17891 hybrid_clock.cc:648] HybridClock initialized: now 1786303275881313 us; error 30 us; skew 500 ppm
Aug 09 19:21:15 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303275, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:15.883697 17891 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:21:15.884075 17891 webserver.cc:460] Webserver started at http://127.9.91.65:35429/ using document root <none> and password file <none>
I20260809 19:21:15.884279 17891 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:15.884326 17891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:15.884449 17891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:15.885368 17891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "0c180550057c4ea4a7483ec83f72c942"
format_stamp: "Formatted at 2026-08-09 19:21:15 on dist-test-slave-1vmg"
server_key: "bb1a547d64dd47674f9f526f43db2676"
server_key_iv: "64893b849a8bfebdd453ed5057224894"
server_key_version: "encryptionkey@0"
I20260809 19:21:15.885684 17891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "0c180550057c4ea4a7483ec83f72c942"
format_stamp: "Formatted at 2026-08-09 19:21:15 on dist-test-slave-1vmg"
server_key: "bb1a547d64dd47674f9f526f43db2676"
server_key_iv: "64893b849a8bfebdd453ed5057224894"
server_key_version: "encryptionkey@0"
I20260809 19:21:15.886837 17891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:21:15.887492 17911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:15.887639 17891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:15.887720 17891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "0c180550057c4ea4a7483ec83f72c942"
format_stamp: "Formatted at 2026-08-09 19:21:15 on dist-test-slave-1vmg"
server_key: "bb1a547d64dd47674f9f526f43db2676"
server_key_iv: "64893b849a8bfebdd453ed5057224894"
server_key_version: "encryptionkey@0"
I20260809 19:21:15.887779 17891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:15.898440 17891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:15.900013 17891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:15.900137 17891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:15.900316 17891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:15.900573 17891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:15.900612 17891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:15.900640 17891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:15.900655 17891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:15.906246 17891 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:43229
I20260809 19:21:15.906302 18086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:43229 every 8 connection(s)
I20260809 19:21:15.906558 17891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:21:15 dist-test-slave-1vmg krb5kdc[17718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303275, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:15.914878 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:15.907249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:53489 (local address 127.9.91.126:46241)
0809 19:21:15.907310 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:15.907312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:15.907477 (+ 165us) server_negotiation.cc:407] Connection header received
0809 19:21:15.908334 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:15.908335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:15.908349 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:15.908403 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:15.908969 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:15.909467 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:15.910404 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:15.910624 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:15.910679 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:15.912593 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:15.912595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:15.912596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:15.912603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:15.913639 (+ 1036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:15.913990 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:15.913991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:15.913991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:15.914060 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:15.914276 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:15.914277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:15.914277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:15.914385 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:15.914442 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:15.914698 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 19:21:15.914788 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:21:15.915032 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:15.907195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46241 (local address 127.9.91.65:53489)
0809 19:21:15.907365 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:15.907398 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:15.908233 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:15.908466 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:15.908469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:15.908707 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:15.908879 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:15.908883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:15.909559 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:15.909560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:15.910320 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:15.910322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:15.910695 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:15.910696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:15.910806 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:15.911211 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:15.911219 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:15.912507 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:15.913710 (+ 1203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:15.913717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:15.913719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:15.913922 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:15.914117 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:15.914118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:15.914119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:15.914219 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:15.914440 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:15.914447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:15.914581 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 19:21:15.914664 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:21:15.914803 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260809 19:21:15.915485 18087 heartbeater.cc:347] Connected to a master server at 127.9.91.126:46241
I20260809 19:21:15.915563 18087 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:15.915730 18087 heartbeater.cc:511] Master 127.9.91.126:46241 requested a full tablet report, sending...
I20260809 19:21:15.915740 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 17891
I20260809 19:21:15.915858 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:15.916051 9581 external_mini_cluster.cc:1428] Setting key 91307e574ef76d4d65b5784569f10c5c
I20260809 19:21:15.916258 17790 ts_manager.cc:194] Registered new tserver with Master: 0c180550057c4ea4a7483ec83f72c942 (127.9.91.65:43229)
I20260809 19:21:15.917033 17790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:53489
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:21:15.939239 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46241
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:37545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:16.017486 18093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:16.017673 18093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:16.017704 18093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:16.018811 18093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:16.018852 18093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:16.018885 18093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:16.020368 18093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:37545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:16.020588 18093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:16.020787 18093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:16.025374 18108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:16.025503 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:16.025583 18103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:16.025709 18093 server_base.cc:1034] running on GCE node
I20260809 19:21:16.025898 18093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:16.026089 18093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:16.027221 18093 hybrid_clock.cc:648] HybridClock initialized: now 1786303276027206 us; error 31 us; skew 500 ppm
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.029600 18093 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:21:16.029960 18093 webserver.cc:460] Webserver started at http://127.9.91.66:43005/ using document root <none> and password file <none>
I20260809 19:21:16.030123 18093 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:16.030162 18093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:16.030247 18093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:16.031185 18093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "a471258283c54d209a816625e58c04d8"
format_stamp: "Formatted at 2026-08-09 19:21:16 on dist-test-slave-1vmg"
server_key: "da649fad157dd8c354feb1dfbce1477c"
server_key_iv: "fcf5d1f64e2d809f9509dbaf66ae0de7"
server_key_version: "encryptionkey@0"
I20260809 19:21:16.031463 18093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "a471258283c54d209a816625e58c04d8"
format_stamp: "Formatted at 2026-08-09 19:21:16 on dist-test-slave-1vmg"
server_key: "da649fad157dd8c354feb1dfbce1477c"
server_key_iv: "fcf5d1f64e2d809f9509dbaf66ae0de7"
server_key_version: "encryptionkey@0"
I20260809 19:21:16.032604 18093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:16.033248 18117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:16.033381 18093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:16.033461 18093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "a471258283c54d209a816625e58c04d8"
format_stamp: "Formatted at 2026-08-09 19:21:16 on dist-test-slave-1vmg"
server_key: "da649fad157dd8c354feb1dfbce1477c"
server_key_iv: "fcf5d1f64e2d809f9509dbaf66ae0de7"
server_key_version: "encryptionkey@0"
I20260809 19:21:16.033517 18093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:16.051326 18093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:16.052944 18093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:16.053076 18093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:16.053272 18093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:16.053550 18093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:16.053586 18093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:16.053611 18093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:16.053637 18093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:16.059347 18093 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:37041
I20260809 19:21:16.059384 18292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:37041 every 8 connection(s)
I20260809 19:21:16.059651 18093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:16.063247 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 18093
I20260809 19:21:16.063351 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:16.063539 9581 external_mini_cluster.cc:1428] Setting key f04eb5873f57f2e97ed49bf596cb6d56
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:16.069237 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:16.060308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:38583 (local address 127.9.91.126:46241)
0809 19:21:16.060366 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.060368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.060525 (+ 157us) server_negotiation.cc:407] Connection header received
0809 19:21:16.061367 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.061369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.061381 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.061452 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.061970 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.062553 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.063854 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.064103 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.064175 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:16.066369 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:16.066371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:16.066371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:16.066379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:16.067813 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.068246 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.068248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.068248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.068320 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.068617 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.068618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.068618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.068756 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:16.068835 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:16.069048 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 19:21:16.069144 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:21:16.069279 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:16.060259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46241 (local address 127.9.91.66:38583)
0809 19:21:16.060416 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:16.060453 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:16.061250 (+ 797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:16.061506 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:16.061508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:16.061735 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:16.061904 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:16.061907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.062643 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.062645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:16.063784 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:16.063786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.064187 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.064188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:16.064320 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:16.064752 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:16.064759 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:16.066274 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:16.067898 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:16.067905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:16.067907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:16.068152 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:16.068394 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:16.068395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:16.068395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:16.068546 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:16.068823 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:16.068826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:16.068931 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 19:21:16.069013 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:21:16.069130 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260809 19:21:16.069731 18295 heartbeater.cc:347] Connected to a master server at 127.9.91.126:46241
I20260809 19:21:16.069825 18295 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:16.069995 18295 heartbeater.cc:511] Master 127.9.91.126:46241 requested a full tablet report, sending...
I20260809 19:21:16.070364 17790 ts_manager.cc:194] Registered new tserver with Master: a471258283c54d209a816625e58c04d8 (127.9.91.66:37041)
I20260809 19:21:16.070858 17790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:38583
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:21:16.087814 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46241
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:37545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:16.166191 18308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:16.166371 18308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:16.166401 18308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:16.167520 18308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:16.167567 18308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:16.167603 18308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:16.169014 18308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:37545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:16.169224 18308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:16.169420 18308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:16.173981 18317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:16.174000 18315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:16.174183 18308 server_base.cc:1034] running on GCE node
W20260809 19:21:16.174026 18320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:16.174451 18308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:16.174677 18308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:16.175810 18308 hybrid_clock.cc:648] HybridClock initialized: now 1786303276175796 us; error 29 us; skew 500 ppm
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.178119 18308 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:21:16.178512 18308 webserver.cc:460] Webserver started at http://127.9.91.67:38483/ using document root <none> and password file <none>
I20260809 19:21:16.178699 18308 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:16.178745 18308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:16.178858 18308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:16.179828 18308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "8a6b07d0b00e486f89e4c4afb608fda0"
format_stamp: "Formatted at 2026-08-09 19:21:16 on dist-test-slave-1vmg"
server_key: "e68a022586ca752597ac80c7667c55e7"
server_key_iv: "c06cb265c2d236f3538df624fdab6c88"
server_key_version: "encryptionkey@0"
I20260809 19:21:16.180150 18308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "8a6b07d0b00e486f89e4c4afb608fda0"
format_stamp: "Formatted at 2026-08-09 19:21:16 on dist-test-slave-1vmg"
server_key: "e68a022586ca752597ac80c7667c55e7"
server_key_iv: "c06cb265c2d236f3538df624fdab6c88"
server_key_version: "encryptionkey@0"
I20260809 19:21:16.181305 18308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:16.181974 18334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:16.182143 18308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:16.182224 18308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "8a6b07d0b00e486f89e4c4afb608fda0"
format_stamp: "Formatted at 2026-08-09 19:21:16 on dist-test-slave-1vmg"
server_key: "e68a022586ca752597ac80c7667c55e7"
server_key_iv: "c06cb265c2d236f3538df624fdab6c88"
server_key_version: "encryptionkey@0"
I20260809 19:21:16.182282 18308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:16.199822 18308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:16.201385 18308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:16.201483 18308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:16.201646 18308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:16.201884 18308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:16.201915 18308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:16.201933 18308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:16.201944 18308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:16.207330 18308 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:40397
I20260809 19:21:16.207389 18509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:40397 every 8 connection(s)
I20260809 19:21:16.207647 18308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:16.212108 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 18308
I20260809 19:21:16.212188 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:16.212363 9581 external_mini_cluster.cc:1428] Setting key cca0280face05f0fbd86aaed4c567fcd
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:16.215746 17850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:16.208285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:43957 (local address 127.9.91.126:46241)
0809 19:21:16.208345 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.208347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.208503 (+ 156us) server_negotiation.cc:407] Connection header received
0809 19:21:16.209295 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.209297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.209306 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.209356 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.209876 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.210361 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.211339 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.211530 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.211588 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:16.213650 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:16.213652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:16.213652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:16.213658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:16.214707 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.215088 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.215089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.215090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.215138 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.215306 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.215306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.215307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.215395 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:16.215435 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:16.215629 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 19:21:16.215685 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:21:16.215899 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:16.208232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46241 (local address 127.9.91.67:43957)
0809 19:21:16.208394 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:16.208431 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:16.209216 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:16.209416 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:16.209419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:16.209650 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:16.209816 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:16.209820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.210419 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.210420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:16.211272 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:16.211275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.211607 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.211608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:16.211698 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:16.212130 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:16.212138 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:16.213568 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:16.214768 (+ 1200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:16.214773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:16.214775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:16.215027 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:16.215181 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:16.215181 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:16.215182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:16.215270 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:16.215432 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:16.215435 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:16.215531 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 19:21:16.215613 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:21:16.215719 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260809 19:21:16.216312 18511 heartbeater.cc:347] Connected to a master server at 127.9.91.126:46241
I20260809 19:21:16.216392 18511 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:16.216548 18511 heartbeater.cc:511] Master 127.9.91.126:46241 requested a full tablet report, sending...
I20260809 19:21:16.216878 17790 ts_manager.cc:194] Registered new tserver with Master: 8a6b07d0b00e486f89e4c4afb608fda0 (127.9.91.67:40397)
I20260809 19:21:16.217211 17790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:43957
I20260809 19:21:16.223944 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/test-admin.keytab.
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.239276 9581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 19:21:16.240753 18519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:16.240032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35300 (local address 127.9.91.65:43229)
0809 19:21:16.240180 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.240184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.240192 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:16.240213 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.240214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.240226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.240288 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.240677 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35300: BlockingRecv error: recv got EOF from 127.0.0.1:35300 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
W20260809 19:21:16.240859 9581 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.241834 9581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.242684 9581 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:21:16.249225 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:16.243152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35304 (local address 127.9.91.65:43229)
0809 19:21:16.243190 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.243192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.243235 (+ 43us) server_negotiation.cc:407] Connection header received
0809 19:21:16.243284 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.243285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.243294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.243337 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.243675 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.244362 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.244997 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.245275 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.245322 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:16.247155 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:16.247162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:16.247163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:16.247169 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:16.248364 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.248644 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.248646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.248646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.248694 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.248812 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.248812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.248813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.248917 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:16.248999 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:16.249063 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 19:21:16.249127 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:21:16.250628 18519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:16.250282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35308 (local address 127.9.91.65:43229)
0809 19:21:16.250315 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.250317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.250329 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:21:16.250370 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.250371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.250378 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.250417 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.250597 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35308: BlockingRecv error: recv got EOF from 127.0.0.1:35308 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:21:16.250749 9581 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.251623 9581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.252441 9581 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:21:16.258131 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:16.252848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35310 (local address 127.9.91.65:43229)
0809 19:21:16.252898 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.252899 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.252951 (+ 52us) server_negotiation.cc:407] Connection header received
0809 19:21:16.252988 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.252989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.252996 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.253037 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.253341 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.253794 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.254384 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.254560 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.254607 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:16.256300 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:16.256302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:16.256302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:16.256307 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:16.257426 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.257665 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.257665 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.257665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.257705 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.257816 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.257816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.257816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.257884 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:16.257939 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:16.258009 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 19:21:16.258061 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 19:21:16.259395 18519 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:16.258999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.9.91.65:43229)
0809 19:21:16.259033 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.259035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.259096 (+ 61us) server_negotiation.cc:407] Connection header received
0809 19:21:16.259138 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.259139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.259146 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.259185 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.259369 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35322: BlockingRecv error: recv got EOF from 127.0.0.1:35322 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:21:16.259570 9581 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786303248613589-9581-0/krb5kdc/krb5cc' of type FILE
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.260392 9581 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:16.261125 9581 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[17718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:21:16.266599 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:16.261526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.9.91.65:43229)
0809 19:21:16.261568 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:16.261569 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:16.261579 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:16.261638 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:16.261639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:16.261646 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:16.261686 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:16.261959 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.262373 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.262995 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:16.263179 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:16.263230 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:16.264921 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:16.264922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:16.264922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:16.264928 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:16.265896 (+ 968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.266153 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.266154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.266154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.266196 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:16.266309 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:16.266309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:16.266310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:16.266377 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:16.266431 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:16.266500 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:21:16.266551 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:21:16.267107 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 17891
I20260809 19:21:16.277626 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 18093
I20260809 19:21:16.288102 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 18308
I20260809 19:21:16.296895 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 17736
2026-08-09T19:21:16Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1909 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[18536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[18536](info): set up 2 sockets
Aug 09 19:21:16 dist-test-slave-1vmg krb5kdc[18536](info): commencing operation
krb5kdc: starting...
W20260809 19:21:17.451784 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:17 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab.
Entry for principal oryx/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab.
I20260809 19:21:17.561513 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:34371
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab
--principal=oryx/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:34371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:17.640122 18558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:17.640305 18558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:17.640334 18558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:17.641424 18558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:17.641463 18558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:17.641482 18558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:17.641498 18558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:17.641512 18558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:17.642962 18558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:34371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab
--principal=oryx/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:34371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:17.643190 18558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:17.643393 18558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:17.647846 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:17.647897 18566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:17.647850 18568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:17.648136 18558 server_base.cc:1034] running on GCE node
I20260809 19:21:17.648365 18558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:17.648631 18558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:17.649773 18558 hybrid_clock.cc:648] HybridClock initialized: now 1786303277649746 us; error 43 us; skew 500 ppm
Aug 09 19:21:17 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:17.652102 18558 init.cc:377] Logged in from keytab as oryx/127.9.91.126@KRBTEST.COM (short username oryx)
I20260809 19:21:17.652513 18558 webserver.cc:460] Webserver started at http://127.9.91.126:37485/ using document root <none> and password file <none>
I20260809 19:21:17.652726 18558 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:17.652772 18558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:17.652894 18558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:17.653836 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "fbb92827f9434fa18ac3d334b7abcd44"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "ac5b59df0d4f03bad2993e843aeac91f"
server_key_iv: "05954a3b6b9ab6c5419914dd9486dbcf"
server_key_version: "encryptionkey@0"
I20260809 19:21:17.654194 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "fbb92827f9434fa18ac3d334b7abcd44"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "ac5b59df0d4f03bad2993e843aeac91f"
server_key_iv: "05954a3b6b9ab6c5419914dd9486dbcf"
server_key_version: "encryptionkey@0"
I20260809 19:21:17.655432 18558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:17.656111 18579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:17.656311 18558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:17.656402 18558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "fbb92827f9434fa18ac3d334b7abcd44"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "ac5b59df0d4f03bad2993e843aeac91f"
server_key_iv: "05954a3b6b9ab6c5419914dd9486dbcf"
server_key_version: "encryptionkey@0"
I20260809 19:21:17.656461 18558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:17.685045 18558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:17.686650 18558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:17.686784 18558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:17.690618 18558 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:34371
I20260809 19:21:17.690662 18657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:34371 every 8 connection(s)
I20260809 19:21:17.690976 18558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:17.691875 18660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:17.693935 18660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: Bootstrap starting.
I20260809 19:21:17.694437 18660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:17.694670 18660 log.cc:824] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:17.695286 18660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: No bootstrap required, opened a new log
I20260809 19:21:17.695456 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 18558
I20260809 19:21:17.695561 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:17.695760 9581 external_mini_cluster.cc:1428] Setting key 867173f527652990f8b314ae10c0e335
I20260809 19:21:17.697189 18660 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } }
I20260809 19:21:17.697331 18660 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:17.697367 18660 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb92827f9434fa18ac3d334b7abcd44, State: Initialized, Role: FOLLOWER
I20260809 19:21:17.697489 18660 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } }
I20260809 19:21:17.697569 18660 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:17.697605 18660 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:17.697664 18660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:17.698559 18660 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } }
I20260809 19:21:17.698668 18660 leader_election.cc:302] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbb92827f9434fa18ac3d334b7abcd44; no voters:
I20260809 19:21:17.698808 18660 leader_election.cc:288] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:17.698839 18669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:17.698959 18669 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 1 LEADER]: Becoming Leader. State: Replica: fbb92827f9434fa18ac3d334b7abcd44, State: Running, Role: LEADER
I20260809 19:21:17.699088 18660 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:17.699112 18669 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } }
I20260809 19:21:17.699468 18669 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbb92827f9434fa18ac3d334b7abcd44. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } }
I20260809 19:21:17.699543 18669 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:17.699645 18672 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } }
I20260809 19:21:17.699725 18672 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
W20260809 19:21:17.701200 18694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:21:17.701510 18694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:21:17.701609 18678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:17.701789 18678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:21:17 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:17.703188 18678 catalog_manager.cc:1347] Generated new cluster ID: 61a364006954433e9f597857f71b3e8c
I20260809 19:21:17.703238 18678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:17.707540 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:17.696253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.9.91.126:34371)
0809 19:21:17.696426 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:17.696435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:17.696458 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:21:17.697162 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:17.697168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:17.697209 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:17.697312 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:17.698089 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:17.699112 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:17.700092 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:17.700653 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:17.700755 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:17.703447 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:17.703451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:17.703451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:17.703462 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:17.705325 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:17.705728 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:17.705731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:17.705731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:17.705863 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:17.706101 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:17.706102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:17.706102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:17.706389 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:17.706988 (+ 599us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:17.707067 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:21:17.707202 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260809 19:21:17.710068 18678 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:17.710528 18678 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:17.721897 18678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: Generated new TSK 0
I20260809 19:21:17.722054 18678 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab.
Entry for principal oryx/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab.
I20260809 19:21:17.751529 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34371
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab
--principal=oryx/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:17.833474 18702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:17.833662 18702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:17.833683 18702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:17.834756 18702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:17.834790 18702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:17.834815 18702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:17.836280 18702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab
--principal=oryx/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:17.836454 18702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:17.836627 18702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:17.841295 18709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:17.841361 18702 server_base.cc:1034] running on GCE node
W20260809 19:21:17.841305 18716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:17.841375 18714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:17.841704 18702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:17.841933 18702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:17.843063 18702 hybrid_clock.cc:648] HybridClock initialized: now 1786303277843052 us; error 30 us; skew 500 ppm
Aug 09 19:21:17 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:17.845466 18702 init.cc:377] Logged in from keytab as oryx/127.9.91.65@KRBTEST.COM (short username oryx)
I20260809 19:21:17.845857 18702 webserver.cc:460] Webserver started at http://127.9.91.65:44027/ using document root <none> and password file <none>
I20260809 19:21:17.846060 18702 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:17.846103 18702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:17.846220 18702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:17.847177 18702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "e6ee6bfb6754406e9e690f6428fa6fba"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "8a95963e9a42421a3791f8a067157e02"
server_key_iv: "b871b1493fe6eec2982f68ce06d30ec8"
server_key_version: "encryptionkey@0"
I20260809 19:21:17.847478 18702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "e6ee6bfb6754406e9e690f6428fa6fba"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "8a95963e9a42421a3791f8a067157e02"
server_key_iv: "b871b1493fe6eec2982f68ce06d30ec8"
server_key_version: "encryptionkey@0"
I20260809 19:21:17.848649 18702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:17.849257 18727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:17.849385 18702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:17.849457 18702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "e6ee6bfb6754406e9e690f6428fa6fba"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "8a95963e9a42421a3791f8a067157e02"
server_key_iv: "b871b1493fe6eec2982f68ce06d30ec8"
server_key_version: "encryptionkey@0"
I20260809 19:21:17.849514 18702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:17.868350 18702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:17.869896 18702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:17.870020 18702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:17.870215 18702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:17.870476 18702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:17.870508 18702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:17.870527 18702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:17.870546 18702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:17.876319 18702 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:42883
I20260809 19:21:17.876356 18910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:42883 every 8 connection(s)
I20260809 19:21:17.876616 18702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:21:17 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:17.884572 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:17.877240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:37569 (local address 127.9.91.126:34371)
0809 19:21:17.877290 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:17.877292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:17.877464 (+ 172us) server_negotiation.cc:407] Connection header received
0809 19:21:17.877916 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:17.877918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:17.877934 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:17.877988 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:17.878839 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:17.879389 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:17.880343 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:17.880561 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:17.880617 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:17.882450 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:17.882452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:17.882453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:17.882459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:17.883578 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:17.883906 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:17.883906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:17.883907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:17.883950 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:17.884135 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:17.884135 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:17.884136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:17.884224 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:17.884267 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:17.884452 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 19:21:17.884511 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:21:17.884727 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:17.877209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.65:37569)
0809 19:21:17.877360 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:17.877393 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:17.877826 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:17.878051 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:17.878055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:17.878578 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:17.878747 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:17.878751 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:17.879472 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:17.879473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:17.880256 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:17.880258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:17.880631 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:17.880632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:17.880749 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:17.881110 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:17.881117 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:17.882369 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:17.883643 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:17.883649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:17.883650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:17.883850 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:17.883992 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:17.883993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:17.883993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:17.884085 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:17.884264 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:17.884266 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:17.884365 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 19:21:17.884433 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:21:17.884544 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260809 19:21:17.885169 18911 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34371
I20260809 19:21:17.885245 18911 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:17.885376 18911 heartbeater.cc:511] Master 127.9.91.126:34371 requested a full tablet report, sending...
I20260809 19:21:17.885794 18604 ts_manager.cc:194] Registered new tserver with Master: e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65:42883)
I20260809 19:21:17.886122 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 18702
I20260809 19:21:17.886222 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:17.886380 9581 external_mini_cluster.cc:1428] Setting key a0bfbc14b06868301dbbd28a4d3f5428
I20260809 19:21:17.886435 18604 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:37569
WARNING: no policy specified for oryx/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab.
Entry for principal oryx/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab.
I20260809 19:21:17.909703 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34371
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab
--principal=oryx/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:17.989171 18918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:17.989356 18918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:17.989391 18918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:17.990487 18918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:17.990532 18918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:17.990571 18918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:17.991978 18918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab
--principal=oryx/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:17.992204 18918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:17.992403 18918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:17.997087 18927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:17.997141 18918 server_base.cc:1034] running on GCE node
W20260809 19:21:17.997073 18926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:17.997048 18929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:17.997494 18918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:17.997723 18918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:17.998853 18918 hybrid_clock.cc:648] HybridClock initialized: now 1786303277998844 us; error 31 us; skew 500 ppm
Aug 09 19:21:17 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.001325 18918 init.cc:377] Logged in from keytab as oryx/127.9.91.66@KRBTEST.COM (short username oryx)
I20260809 19:21:18.001761 18918 webserver.cc:460] Webserver started at http://127.9.91.66:39277/ using document root <none> and password file <none>
I20260809 19:21:18.001950 18918 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.001995 18918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.002111 18918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:18.003120 18918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "0195befba7b040dfaed2d0055fc2405e"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "fe0f3c45a700c6bdd8df70f3a9fca412"
server_key_iv: "a92d8ab19f40c03655d4c26259101730"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.003424 18918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "0195befba7b040dfaed2d0055fc2405e"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "fe0f3c45a700c6bdd8df70f3a9fca412"
server_key_iv: "a92d8ab19f40c03655d4c26259101730"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.004699 18918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:18.005319 18942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.005472 18918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:18.005548 18918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "0195befba7b040dfaed2d0055fc2405e"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "fe0f3c45a700c6bdd8df70f3a9fca412"
server_key_iv: "a92d8ab19f40c03655d4c26259101730"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.005606 18918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.023094 18918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.024732 18918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:18.024860 18918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.025048 18918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:18.025311 18918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:18.025343 18918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.025368 18918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:18.025385 18918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.030817 18918 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:44323
I20260809 19:21:18.030871 19114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:44323 every 8 connection(s)
I20260809 19:21:18.031139 18918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:18.033546 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 18918
I20260809 19:21:18.033644 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:18.033836 9581 external_mini_cluster.cc:1428] Setting key d425166f8d2aec97f2f55ad983d68e38
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:18.040364 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.031758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:57661 (local address 127.9.91.126:34371)
0809 19:21:18.031816 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.031818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.031987 (+ 169us) server_negotiation.cc:407] Connection header received
0809 19:21:18.032420 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.032421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.032435 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.032486 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.033311 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.033854 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.035025 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.035233 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.035282 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.037460 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.037462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.037462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.037470 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.039076 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.039458 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.039459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.039460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.039531 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.039755 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.039756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.039756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.039898 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.039970 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.040187 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.040272 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:21:18.040402 19119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.031737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.66:57661)
0809 19:21:18.031881 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.031912 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.032323 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.032544 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.032547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.033051 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.033216 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.033220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.033943 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.033944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.034938 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.034942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.035307 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.035308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.035426 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.035846 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.035853 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.037368 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.039165 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.039173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.039174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.039387 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.039594 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.039594 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.039595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.039683 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.039958 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.039961 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.040070 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.040150 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.040261 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260809 19:21:18.040856 19116 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34371
I20260809 19:21:18.040944 19116 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.041107 19116 heartbeater.cc:511] Master 127.9.91.126:34371 requested a full tablet report, sending...
I20260809 19:21:18.041486 18604 ts_manager.cc:194] Registered new tserver with Master: 0195befba7b040dfaed2d0055fc2405e (127.9.91.66:44323)
I20260809 19:21:18.041857 18604 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:57661
WARNING: no policy specified for oryx/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab.
Entry for principal oryx/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab.
I20260809 19:21:18.057174 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34371
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab
--principal=oryx/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:18.134708 19124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:18.134869 19124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:18.134899 19124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:18.135993 19124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:18.136032 19124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:18.136065 19124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:18.137414 19124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab
--principal=oryx/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:34371
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:18.137619 19124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:18.137820 19124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.142367 19137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.142436 19135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.142447 19134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.142423 19124 server_base.cc:1034] running on GCE node
I20260809 19:21:18.142740 19124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.142987 19124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.144140 19124 hybrid_clock.cc:648] HybridClock initialized: now 1786303278144124 us; error 33 us; skew 500 ppm
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.146370 19124 init.cc:377] Logged in from keytab as oryx/127.9.91.67@KRBTEST.COM (short username oryx)
I20260809 19:21:18.146745 19124 webserver.cc:460] Webserver started at http://127.9.91.67:38333/ using document root <none> and password file <none>
I20260809 19:21:18.146906 19124 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.147001 19124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.147099 19124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:18.148018 19124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "d1bba31429c34198b9b265c107d9314d"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "eb3c1e4b0aac61bbdfa8a1f635080862"
server_key_iv: "c97adb73ca411f71d4f4eba54e94e8f7"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.148314 19124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "d1bba31429c34198b9b265c107d9314d"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "eb3c1e4b0aac61bbdfa8a1f635080862"
server_key_iv: "c97adb73ca411f71d4f4eba54e94e8f7"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.149523 19124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:21:18.150156 19146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.150295 19124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:18.150379 19124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "d1bba31429c34198b9b265c107d9314d"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "eb3c1e4b0aac61bbdfa8a1f635080862"
server_key_iv: "c97adb73ca411f71d4f4eba54e94e8f7"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.150442 19124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.158797 19124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.160542 19124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:18.160674 19124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.160876 19124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:18.161162 19124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:18.161201 19124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.161229 19124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:18.161257 19124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.167487 19124 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:41079
I20260809 19:21:18.167564 19319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:41079 every 8 connection(s)
I20260809 19:21:18.167811 19124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:18.171223 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 19124
I20260809 19:21:18.171314 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:18.171478 9581 external_mini_cluster.cc:1428] Setting key c116346120864b91f5828bdc1f222248
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:18.176105 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.168401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:49249 (local address 127.9.91.126:34371)
0809 19:21:18.168477 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.168479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.168621 (+ 142us) server_negotiation.cc:407] Connection header received
0809 19:21:18.169024 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.169025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.169036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.169079 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.169887 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.170369 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.171344 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.171552 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.171621 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.173603 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.173604 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.173605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.173612 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.174723 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.175158 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.175159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.175159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.175226 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.175463 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.175464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.175465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.175584 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.175649 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.175903 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.176007 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:21:18.176214 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.168353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.67:49249)
0809 19:21:18.168513 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.168546 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.168949 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.169127 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.169130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.169648 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.169816 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.169820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.170430 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.170431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.171259 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.171262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.171626 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.171628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.171762 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.172232 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.172239 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.173510 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.174800 (+ 1290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.174807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.174809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.175087 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.175294 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.175295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.175295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.175403 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.175646 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.175653 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.175780 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.175871 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.176018 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260809 19:21:18.176653 19320 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34371
I20260809 19:21:18.176733 19320 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.176898 19320 heartbeater.cc:511] Master 127.9.91.126:34371 requested a full tablet report, sending...
I20260809 19:21:18.177246 18604 ts_manager.cc:194] Registered new tserver with Master: d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079)
I20260809 19:21:18.177598 18604 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:49249
I20260809 19:21:18.183032 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.91.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.125.keytab.
Entry for principal oryx/127.9.91.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.125.keytab.
I20260809 19:21:18.207237 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:38385
--webserver_interface=127.9.91.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.125.keytab
--principal=oryx/127.9.91.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:34371,127.9.91.125:38385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:18.288151 19329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:18.288496 19329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:18.288573 19329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:18.290086 19329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:18.290153 19329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:18.290184 19329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:18.290212 19329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:18.290234 19329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:18.292040 19329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:34371,127.9.91.125:38385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.125.keytab
--principal=oryx/127.9.91.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:38385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.91.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
W20260809 19:21:18.292166 19329 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.91.126:34371,127.9.91.125:38385'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 19:21:18.292300 19329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:18.292495 19329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.296808 19339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.296845 19341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.296792 19343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.296986 19329 server_base.cc:1034] running on GCE node
I20260809 19:21:18.297258 19329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.297518 19329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.298655 19329 hybrid_clock.cc:648] HybridClock initialized: now 1786303278298642 us; error 25 us; skew 500 ppm
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.301189 19329 init.cc:377] Logged in from keytab as oryx/127.9.91.125@KRBTEST.COM (short username oryx)
I20260809 19:21:18.301535 19329 webserver.cc:460] Webserver started at http://127.9.91.125:34179/ using document root <none> and password file <none>
I20260809 19:21:18.301712 19329 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.301748 19329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.301858 19329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:18.302795 19329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/instance:
uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "86d2d7b023f65e2c811f2193fe735aab"
server_key_iv: "f5e911aa7e74e236ae32f0f866285f80"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.303123 19329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal/instance:
uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "86d2d7b023f65e2c811f2193fe735aab"
server_key_iv: "f5e911aa7e74e236ae32f0f866285f80"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.304293 19329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:18.304917 19352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.305091 19329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.305176 19329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "86d2d7b023f65e2c811f2193fe735aab"
server_key_iv: "f5e911aa7e74e236ae32f0f866285f80"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.305228 19329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.318591 19329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.320147 19329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:18.320251 19329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.323882 19329 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.125:38385
I20260809 19:21:18.323928 19429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.125:38385 every 8 connection(s)
I20260809 19:21:18.324194 19329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
I20260809 19:21:18.325050 19430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:18.326382 19430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.125@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:18.331324 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 19329
I20260809 19:21:18.334260 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.326980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36554 (local address 127.9.91.126:34371)
0809 19:21:18.327038 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.327040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.327225 (+ 185us) server_negotiation.cc:407] Connection header received
0809 19:21:18.327663 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.327665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.327677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.327729 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.328262 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.328742 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.329644 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.329847 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.329902 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.331778 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.331779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.331780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.331788 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.333021 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.333370 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.333371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.333372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.333426 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.333634 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.333635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.333635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.333783 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.333839 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.334081 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.334171 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:21:18.334381 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.326945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:36554)
0809 19:21:18.327123 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.327156 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.327566 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.327787 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.327791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.328002 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.328178 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.328181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.328821 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.328821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.329563 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.329565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.329915 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.329915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.330021 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.330384 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.330391 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.331687 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.333098 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.333105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.333107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.333305 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.333479 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.333480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.333481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.333579 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.333839 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.333846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.333968 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.334051 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.334190 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260809 19:21:18.334932 19430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.91.125@KRBTEST.COM
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.125@KRBTEST.COM for oryx/127.9.91.125@KRBTEST.COM
I20260809 19:21:18.339645 19435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.331829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55264 (local address 127.9.91.125:38385)
0809 19:21:18.331982 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.331986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.332043 (+ 57us) server_negotiation.cc:407] Connection header received
0809 19:21:18.332106 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.332109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.332121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.332177 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.332723 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.333359 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.334048 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.334339 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.334402 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.336781 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.336788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.336788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.336794 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.338320 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.338658 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.338661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.338662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.338720 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.338934 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.338935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.338936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.339131 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.339264 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.339456 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.339540 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260809 19:21:18.341302 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.335146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.0.0.1:55276)
0809 19:21:18.335182 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.335190 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.335223 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.335430 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.335430 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.335557 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.335598 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.335599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.336165 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.336166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.336803 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.336806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.337137 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.337138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.337199 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.337498 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.337501 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.338642 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.340647 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.340648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.340649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.340795 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.340928 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.340928 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.340928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.340969 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.341114 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.341114 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.341146 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.341185 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.341249 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 19:21:18.341403 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.335218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55276 (local address 127.9.91.125:38385)
0809 19:21:18.335344 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.335346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.335351 (+ 5us) server_negotiation.cc:407] Connection header received
0809 19:21:18.335363 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.335364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.335370 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.335422 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.335660 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.336103 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.336857 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.337076 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.337124 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.338708 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.338709 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.338710 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.338715 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.340584 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.340847 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.340848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.340848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.340888 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.340999 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.340999 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.341000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.341075 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.341114 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.341205 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.341292 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260809 19:21:18.342373 19430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Bootstrap starting.
I20260809 19:21:18.342902 19430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:18.343226 19430 log.cc:824] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:18.343878 19430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: No bootstrap required, opened a new log
I20260809 19:21:18.345111 19430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } }
I20260809 19:21:18.345213 19430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:18.345239 19430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508f35eb0fbb425b9c0f975d8ce0bcb4, State: Initialized, Role: FOLLOWER
I20260809 19:21:18.345326 19430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } }
I20260809 19:21:18.345566 19430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:18.345546 19439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } } }
I20260809 19:21:18.345610 19439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:21:18.347062 19461 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:21:18.347118 19461 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:21:18.348460 19329 master_runner.cc:428] Detected that this master 508f35eb0fbb425b9c0f975d8ce0bcb4 is joining an existing cluster
I20260809 19:21:18.348501 19329 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.91.125:38385
I20260809 19:21:18.355569 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.349337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36564 (local address 127.9.91.126:34371)
0809 19:21:18.349485 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.349487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.349495 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:18.349577 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.349578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.349586 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.349613 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.349940 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.350665 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.351642 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.351910 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.351966 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.352998 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.353000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.353000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.353007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.354425 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.354789 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.354790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.354791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.354844 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.355064 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.355065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.355065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.355186 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.355217 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.355425 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.355496 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 19:21:18.355966 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.349327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.0.0.1:55288)
0809 19:21:18.349478 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.349487 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.349536 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.349639 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.349640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.349816 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.349887 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.349889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.350746 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.350747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.351681 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.351684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.352093 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.352094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.352159 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.352560 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.352564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.353049 (+ 485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.354639 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.354640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.354640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.354863 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.355108 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.355109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.355109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.355184 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.355429 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.355430 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.355478 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.355547 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.355843 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260809 19:21:18.356055 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.349522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.9.91.125:38385)
0809 19:21:18.349551 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.349553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.349559 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:21:18.349577 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.349577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.349585 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.349633 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.349978 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.350665 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.351751 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.352025 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.352082 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.353149 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.353150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.353151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.353157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.354567 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.354984 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.354985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.354986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.355049 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.355257 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.355258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.355258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.355366 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.355426 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.355577 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.355902 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":250,"server-negotiator.queue_time_us":17}
I20260809 19:21:18.356146 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.349303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:36564)
0809 19:21:18.349447 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.349457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.349507 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.349680 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.349681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.349819 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.349873 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.349874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.350962 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.350962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.351539 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.351542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.352011 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.352013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.352067 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.352416 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.352421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.352884 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.354514 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.354515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.354515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.354730 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.354895 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.354896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.354896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.354985 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.355260 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.355261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.355311 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.355645 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.355969 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":233,"thread_start_us":93,"threads_started":1}
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.126@KRBTEST.COM for oryx/127.9.91.125@KRBTEST.COM
I20260809 19:21:18.364418 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.358197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55296 (local address 127.9.91.125:38385)
0809 19:21:18.358236 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.358238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.358385 (+ 147us) server_negotiation.cc:407] Connection header received
0809 19:21:18.358450 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.358451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.358460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.358501 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.358884 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.359362 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.360075 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.360295 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.360348 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.362244 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.362245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.362245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.362251 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.363439 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.363835 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.363835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.363835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.363875 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.364046 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.364046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.364047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.364122 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.364165 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.364321 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.364370 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:21:18.364575 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.358141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.0.0.1:55296)
0809 19:21:18.358287 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.358325 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.358390 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.358515 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.358519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.358764 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.358808 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.358810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.359429 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.359430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.360001 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.360003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.360351 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.360352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.360426 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.360792 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.360795 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.362170 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.363512 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.363519 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.363522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.363769 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.363925 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.363926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.363926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.363989 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.364172 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.364178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.364226 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.364307 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.364415 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260809 19:21:18.365064 18604 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.91.125:38385: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4"
member_type: NON_VOTER
last_known_addr {
host: "127.9.91.125"
port: 38385
}
attrs {
promote: true
}
}
dest_uuid: "fbb92827f9434fa18ac3d334b7abcd44"
cas_config_opid_index: -1
I20260809 19:21:18.365372 18604 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } }
I20260809 19:21:18.365798 19477 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 508f35eb0fbb425b9c0f975d8ce0bcb4 (127.9.91.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } }
I20260809 19:21:18.366044 19393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:18.366604 19477 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.91.125:38385
I20260809 19:21:18.366686 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } }
I20260809 19:21:18.366750 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:18.366844 19393 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Refusing update from remote peer fbb92827f9434fa18ac3d334b7abcd44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 19:21:18.367239 19329 master.cc:537] Master@127.9.91.125:38385 shutting down...
I20260809 19:21:18.367244 19477 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:21:18.367913 18580 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 19:21:18.368741 18580 consensus_peers.cc:487] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 -> Peer 508f35eb0fbb425b9c0f975d8ce0bcb4 (127.9.91.125:38385): Couldn't send request to peer 508f35eb0fbb425b9c0f975d8ce0bcb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:18.369860 19329 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:18.369938 19329 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:18.369982 19329 tablet_replica.cc:330] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: stopping tablet replica
I20260809 19:21:18.382350 19329 master.cc:552] Master@127.9.91.125:38385 shutdown complete.
I20260809 19:21:18.382386 19329 master_runner.cc:305] Clearing existing system tablet
I20260809 19:21:18.382486 19329 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.382529 19329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.383054 19329 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.383445 19483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.383539 19329 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.383589 19329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.383611 19329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "86d2d7b023f65e2c811f2193fe735aab"
server_key_iv: "f5e911aa7e74e236ae32f0f866285f80"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.384526 19329 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:18.385531 19329 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 19:21:18.385586 19329 log.cc:1196] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 19:21:18.385744 19329 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Deleting consensus metadata
I20260809 19:21:18.385859 19329 master_runner.cc:315] Copying system tablet from 127.9.91.126:34371
I20260809 19:21:18.386368 19329 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: tablet copy: Beginning tablet copy session from remote peer at address 127.9.91.126:34371
I20260809 19:21:18.391515 19492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.386633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:36572)
0809 19:21:18.386743 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.386753 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.386809 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.386932 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.386932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.387068 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.387112 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.387113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.387683 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.387684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.388201 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.388202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.388515 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.388516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.388577 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.388970 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.388973 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.389214 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.390514 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.390515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.390516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.390704 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.390978 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.390978 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.390979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.391037 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.391297 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.391298 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.391340 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.391384 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.391452 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
I20260809 19:21:18.391599 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.386673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36572 (local address 127.9.91.126:34371)
0809 19:21:18.386708 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.386710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.386806 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:21:18.386851 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.386851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.386860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.386900 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.387183 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.387611 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.388271 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.388459 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.388509 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.389287 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.389289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.389289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.389295 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.390449 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.390770 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.390771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.390772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.390842 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.391099 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.391099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.391100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.391233 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.391311 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.391424 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.391510 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:21:18.391907 18632 tablet_copy_service.cc:140] P fbb92827f9434fa18ac3d334b7abcd44: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 508f35eb0fbb425b9c0f975d8ce0bcb4 ({username='oryx', principal='oryx/127.9.91.125@KRBTEST.COM'} at 127.0.0.1:36572)
I20260809 19:21:18.391976 18632 tablet_copy_service.cc:161] P fbb92827f9434fa18ac3d334b7abcd44: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 508f35eb0fbb425b9c0f975d8ce0bcb4 at {username='oryx', principal='oryx/127.9.91.125@KRBTEST.COM'} at 127.0.0.1:36572: session id = 508f35eb0fbb425b9c0f975d8ce0bcb4-00000000000000000000000000000000
I20260809 19:21:18.392475 18632 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 19:21:18.392988 19329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 19:21:18.393692 19329 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: tablet copy: Starting download of 0 data blocks...
I20260809 19:21:18.393797 19329 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: tablet copy: Starting download of 1 WAL segments...
I20260809 19:21:18.394747 19329 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 19:21:18.395462 18632 tablet_copy_service.cc:342] P fbb92827f9434fa18ac3d334b7abcd44: Request end of tablet copy session 508f35eb0fbb425b9c0f975d8ce0bcb4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.91.125@KRBTEST.COM'} at 127.0.0.1:36572
I20260809 19:21:18.395512 18632 tablet_copy_service.cc:433] P fbb92827f9434fa18ac3d334b7abcd44: ending tablet copy session 508f35eb0fbb425b9c0f975d8ce0bcb4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 508f35eb0fbb425b9c0f975d8ce0bcb4
I20260809 19:21:18.397464 19329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.398368 19500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.398526 19493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.398577 19496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.399070 19329 server_base.cc:1034] running on GCE node
I20260809 19:21:18.399166 19329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.399274 19329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.400380 19329 hybrid_clock.cc:648] HybridClock initialized: now 1786303278400371 us; error 25 us; skew 500 ppm
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.401592 19329 init.cc:377] Logged in from keytab as oryx/127.9.91.125@KRBTEST.COM (short username oryx)
I20260809 19:21:18.401844 19329 webserver.cc:460] Webserver started at http://127.9.91.125:38427/ using document root <none> and password file <none>
I20260809 19:21:18.401958 19329 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.401997 19329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.402678 19329 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:18.403246 19510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.403348 19329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.403390 19329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "86d2d7b023f65e2c811f2193fe735aab"
server_key_iv: "f5e911aa7e74e236ae32f0f866285f80"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.403432 19329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.410588 19329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.411902 19329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.414671 19329 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.125:38385
I20260809 19:21:18.415055 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.125:38385 every 8 connection(s)
I20260809 19:21:18.415079 19329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
I20260809 19:21:18.415849 19590 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:21:18.416320 19590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Bootstrap starting.
I20260809 19:21:18.419142 19590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:21:18.419385 19590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Bootstrap complete.
I20260809 19:21:18.419669 19590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } }
I20260809 19:21:18.419766 19590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 508f35eb0fbb425b9c0f975d8ce0bcb4, State: Initialized, Role: LEARNER
I20260809 19:21:18.419816 19590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } }
I20260809 19:21:18.419978 19590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:18.419986 19596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } }
I20260809 19:21:18.420096 19596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:21:18.421439 19614 catalog_manager.cc:1259] Loaded cluster ID: 61a364006954433e9f597857f71b3e8c
I20260809 19:21:18.421835 19614 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: loading cluster ID for follower catalog manager: success
I20260809 19:21:18.422526 19614 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: acquiring CA information for follower catalog manager: success
I20260809 19:21:18.422751 19614 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:21:18.447216 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.441061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.9.91.125:38385)
0809 19:21:18.441167 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.441168 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.441178 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:18.441198 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.441198 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.441208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.441279 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.441731 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.442735 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.443440 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.443737 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.443805 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.444785 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.444786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.444787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.444793 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.446200 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.446495 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.446496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.446496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.446565 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.446721 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.446722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.446722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.446838 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.446951 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.447027 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.447104 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":48,"threads_started":1}
WARNING: no policy specified for oryx/127.9.91.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.124.keytab.
Entry for principal oryx/127.9.91.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.124.keytab.
I20260809 19:21:18.471460 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:38053
--webserver_interface=127.9.91.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.124.keytab
--principal=oryx/127.9.91.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:18.552122 19623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:18.552300 19623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:18.552323 19623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:18.553691 19623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:18.553740 19623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:18.553754 19623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:18.553766 19623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:18.553777 19623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:18.555320 19623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.124.keytab
--principal=oryx/127.9.91.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:38053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.91.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:18.555529 19623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:18.555740 19623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.560273 19633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.560338 19631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.560281 19637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.560487 19623 server_base.cc:1034] running on GCE node
I20260809 19:21:18.560739 19623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.561002 19623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.562142 19623 hybrid_clock.cc:648] HybridClock initialized: now 1786303278562129 us; error 31 us; skew 500 ppm
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.564580 19623 init.cc:377] Logged in from keytab as oryx/127.9.91.124@KRBTEST.COM (short username oryx)
I20260809 19:21:18.564962 19623 webserver.cc:460] Webserver started at http://127.9.91.124:33969/ using document root <none> and password file <none>
I20260809 19:21:18.565150 19623 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.565192 19623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.565287 19623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:18.566320 19623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/instance:
uuid: "da147739e9334c93922217ef735cd93a"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "c6557b13bd63e3d5930fceba1d9eaf24"
server_key_iv: "dc1894ca5818c64731b9d66d8f5185e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.566625 19623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal/instance:
uuid: "da147739e9334c93922217ef735cd93a"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "c6557b13bd63e3d5930fceba1d9eaf24"
server_key_iv: "dc1894ca5818c64731b9d66d8f5185e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.567991 19623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:21:18.568708 19646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.568876 19623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:18.568964 19623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
uuid: "da147739e9334c93922217ef735cd93a"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "c6557b13bd63e3d5930fceba1d9eaf24"
server_key_iv: "dc1894ca5818c64731b9d66d8f5185e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.569022 19623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.579910 19623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.581473 19623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:18.581591 19623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.587236 19623 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.124:38053
I20260809 19:21:18.587704 19623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
I20260809 19:21:18.589793 19736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.124:38053 every 8 connection(s)
I20260809 19:21:18.591007 19739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:18.592334 19739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:21:18.597621 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 19623
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.124@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:18.602090 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.593038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36574 (local address 127.9.91.126:34371)
0809 19:21:18.593103 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.593105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.594353 (+ 1248us) server_negotiation.cc:407] Connection header received
0809 19:21:18.594817 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.594819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.594832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.594882 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.595447 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.595932 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.596858 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.597046 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.597105 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.599298 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.599300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.599301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.599309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.600807 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.601148 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.601149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.601149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.601201 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.601421 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.601422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.601422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.601535 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.601595 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.601918 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.602000 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:21:18.602552 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.592882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:36574)
0809 19:21:18.594152 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.594184 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.594726 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.594958 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.594962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.595186 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.595362 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.595366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.596011 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.596012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.596777 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.596779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.597118 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.597119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.597240 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.597654 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.597662 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.599221 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.600868 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.600874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.600877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.601077 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.601265 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.601266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.601267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.601365 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.601600 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.601608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.601766 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.601887 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.602042 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1235,"thread_start_us":83,"threads_started":1}
I20260809 19:21:18.603892 19739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.91.124@KRBTEST.COM
I20260809 19:21:18.607498 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.598168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.9.91.124:38053)
0809 19:21:18.598378 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.598383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.598405 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:21:18.598424 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.598426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.598444 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.598575 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.599454 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.600497 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.601553 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.601897 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.602257 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.604696 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.604703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.604703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.604708 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.606144 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.606555 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.606557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.606558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.606617 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.606847 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.606848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.606848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.607079 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.607260 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.607309 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.607388 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":146,"threads_started":1}
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.124@KRBTEST.COM for oryx/127.9.91.125@KRBTEST.COM
I20260809 19:21:18.611265 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.604146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.0.0.1:55318)
0809 19:21:18.604727 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.604738 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.604785 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.604970 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.604971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.605085 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.605160 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.605162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.606085 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.606086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.606666 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.606668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.607048 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.607049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.607120 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.607563 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.607568 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.609016 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.610385 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.610386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.610386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.610536 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.610706 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.610706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.610707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.610761 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.610996 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.610996 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.611037 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.611098 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.611183 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562}
I20260809 19:21:18.611299 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.604197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.9.91.125:38385)
0809 19:21:18.604234 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.604236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.604788 (+ 552us) server_negotiation.cc:407] Connection header received
0809 19:21:18.604856 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.604857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.604871 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.604924 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.605256 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.605991 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.606728 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.606978 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.607032 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.609101 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.609102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.609102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.609109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.610314 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.610596 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.610597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.610597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.610653 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.610811 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.610812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.610812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.610945 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.611005 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.611135 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.611216 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:21:18.611572 19739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.124@KRBTEST.COM for oryx/127.9.91.124@KRBTEST.COM
I20260809 19:21:18.617722 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.611761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.0.0.1:36298)
0809 19:21:18.611799 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.611809 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.611843 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.611983 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.611983 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.612079 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.612104 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.612105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.612902 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.612903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.613339 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.613340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.613793 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.613793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.613825 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.614165 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.614168 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.615333 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.616938 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.616939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.616939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.617085 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.617267 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.617268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.617268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.617309 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.617531 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.617531 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.617560 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.617601 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.617671 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:21:18.617823 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.611812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.9.91.124:38053)
0809 19:21:18.611848 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.611850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.611887 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:21:18.611908 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.611909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.611920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.611982 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.612165 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.612824 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.613405 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.613714 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.613799 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.615418 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.615419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.615420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.615427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.616861 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.617147 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.617148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.617149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.617209 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.617345 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.617345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.617346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.617460 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.617542 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.617627 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.617735 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":15}
I20260809 19:21:18.619163 19739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Bootstrap starting.
I20260809 19:21:18.620208 19739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:18.620568 19739 log.cc:824] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:18.621465 19739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: No bootstrap required, opened a new log
I20260809 19:21:18.623636 19739 raft_consensus.cc:348] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } }
I20260809 19:21:18.623790 19739 raft_consensus.cc:374] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:18.623834 19739 raft_consensus.cc:729] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da147739e9334c93922217ef735cd93a, State: Initialized, Role: FOLLOWER
I20260809 19:21:18.623960 19739 consensus_queue.cc:260] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } }
I20260809 19:21:18.624277 19744 sys_catalog.cc:455] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } } }
I20260809 19:21:18.624336 19739 sys_catalog.cc:564] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:18.624361 19744 sys_catalog.cc:458] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:21:18.626506 19760 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:21:18.626559 19760 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:21:18.628861 19623 master_runner.cc:428] Detected that this master da147739e9334c93922217ef735cd93a is joining an existing cluster
I20260809 19:21:18.628913 19623 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.91.124:38053
I20260809 19:21:18.637550 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.630421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.9.91.125:38385)
0809 19:21:18.630467 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.630469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.630885 (+ 416us) server_negotiation.cc:407] Connection header received
0809 19:21:18.630983 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.630985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.630995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.631054 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.631401 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.632094 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.633176 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.633456 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.633528 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.634493 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.634495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.634495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.634502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.636313 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.636705 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.636706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.636706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.636781 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.636984 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.636985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.636985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.637104 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.637160 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.637391 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.637468 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:21:18.638036 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.630119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.0.0.1:55324)
0809 19:21:18.630810 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.630819 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.630883 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.631059 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.631060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.631251 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.631327 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.631329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.632187 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.632188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.633087 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.633089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.633555 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.633556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.633619 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.634019 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.634024 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.634400 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.636400 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.636401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.636401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.636621 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.636850 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.636850 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.636850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.636919 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.637209 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.637209 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.637252 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.637833 (+ 581us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.637935 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":162,"threads_started":1}
I20260809 19:21:18.655810 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.630207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.9.91.126:34371)
0809 19:21:18.630246 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.630248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.635071 (+ 4823us) server_negotiation.cc:407] Connection header received
0809 19:21:18.635140 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.635141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.635151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.635205 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.635899 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.636573 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.637617 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.637898 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.637938 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.639025 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.639027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.639027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.639035 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.640971 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.641646 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.641648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.641648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.641730 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.655185 (+ 13455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.655187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.655187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.655326 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.655375 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.655635 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.655715 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:21:18.656025 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.630611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.0.0.1:36312)
0809 19:21:18.635394 (+ 4783us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.635404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.635467 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.635637 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.635638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.635806 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.636221 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.636224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.639021 (+ 2797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.639021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.639838 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.639841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.640303 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.640304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.640357 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.640721 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.640726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.641071 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.643011 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.643012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.643012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.655273 (+ 12261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.655460 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.655461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.655461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.655505 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.655740 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.655741 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.655772 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.655817 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.655902 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4766,"thread_start_us":122,"threads_started":1}
I20260809 19:21:18.656329 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.630682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.9.91.124:38053)
0809 19:21:18.630747 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.630749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.635468 (+ 4719us) server_negotiation.cc:407] Connection header received
0809 19:21:18.635552 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.635553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.635563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.635620 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.636433 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.637088 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.639925 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.640213 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.640246 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.641165 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.641166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.641167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.641172 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.642888 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.655357 (+ 12469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.655358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.655358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.655406 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.655555 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.655556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.655556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.655669 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.655722 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.656188 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.656255 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 19:21:18.656844 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.630308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:36584)
0809 19:21:18.635001 (+ 4693us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.635012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.635079 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.635249 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.635250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.635730 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.635792 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.635793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.636672 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.636673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.637508 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.637511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.638014 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.638014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.638078 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.638506 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.638511 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.638892 (+ 381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.641074 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.641075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.641076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.641532 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.642130 (+ 598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.642130 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.642131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.655073 (+ 12942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.655431 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.655432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.655506 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.656300 (+ 794us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.656726 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4655,"thread_start_us":72,"threads_started":1}
I20260809 19:21:18.658869 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 18702
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303277, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.126@KRBTEST.COM for oryx/127.9.91.124@KRBTEST.COM
I20260809 19:21:18.675912 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.659024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.0.0.1:36316)
0809 19:21:18.664226 (+ 5202us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.664236 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.664279 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.668119 (+ 3840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.668121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.668323 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.668371 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.668374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.669879 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.669880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.670691 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.670694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.671229 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.671231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.671298 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.671669 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.671674 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.673163 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.674852 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.674853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.674854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.675093 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.675289 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.675289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.675290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.675344 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.675594 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.675595 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.675651 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.675710 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.675816 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5184}
I20260809 19:21:18.676440 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.667065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.9.91.124:38053)
0809 19:21:18.667923 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.667925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.667934 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:18.667967 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.667967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.667977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.668053 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.668863 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.669788 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.670812 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.671123 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.671180 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.673254 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.673256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.673256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.673262 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.674765 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.675170 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.675171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.675171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.675225 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.675403 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.675404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.675404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.675517 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.675569 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.676281 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.676356 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838}
I20260809 19:21:18.676983 18604 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.91.124:38053: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "da147739e9334c93922217ef735cd93a"
member_type: NON_VOTER
last_known_addr {
host: "127.9.91.124"
port: 38053
}
attrs {
promote: true
}
}
dest_uuid: "fbb92827f9434fa18ac3d334b7abcd44"
cas_config_opid_index: 5
I20260809 19:21:18.677189 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:42883
--local_ip_for_outbound_sockets=127.9.91.65
--tserver_master_addrs=127.9.91.126:34371
--webserver_port=44027
--webserver_interface=127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab
--principal=oryx/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:21:18.677208 18604 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } }
I20260809 19:21:18.677686 19477 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER da147739e9334c93922217ef735cd93a (127.9.91.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.678581 19477 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.91.124:38053
I20260809 19:21:18.678663 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.678728 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:18.679396 19623 master.cc:537] Master@127.9.91.124:38053 shutting down...
I20260809 19:21:18.679723 19692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:18.680645 19692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Refusing update from remote peer fbb92827f9434fa18ac3d334b7abcd44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 19:21:18.680905 19476 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:21:18.681552 18582 consensus_peers.cc:487] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 -> Peer da147739e9334c93922217ef735cd93a (127.9.91.124:38053): Couldn't send request to peer da147739e9334c93922217ef735cd93a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:18.683904 19623 raft_consensus.cc:2229] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:18.683986 19623 raft_consensus.cc:2258] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:18.684026 19623 tablet_replica.cc:330] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: stopping tablet replica
I20260809 19:21:18.688292 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.679136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55334 (local address 127.9.91.125:38385)
0809 19:21:18.679171 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.679173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.683342 (+ 4169us) server_negotiation.cc:407] Connection header received
0809 19:21:18.683439 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.683440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.683464 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.683501 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:18.684989 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.685721 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.687129 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.687734 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.687781 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.687876 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.688186 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.688226 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:18.688620 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.679051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.0.0.1:55334)
0809 19:21:18.683283 (+ 4232us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.683295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.683369 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.684869 (+ 1500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.684871 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.684873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:18.684906 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.684907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.685815 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.685816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.687029 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.687032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.688017 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.688018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.688086 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.688108 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.688515 (+ 407us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.688559 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4215}
I20260809 19:21:18.689004 19551 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 LEARNER]: Refusing update from remote peer fbb92827f9434fa18ac3d334b7abcd44: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 19:21:18.689285 19476 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 19:21:18.689800 19551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER da147739e9334c93922217ef735cd93a (127.9.91.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.690791 19596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbb92827f9434fa18ac3d334b7abcd44. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.690871 19596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:21:18.690899 19596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: NON_VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: true } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.690953 19596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: LEARNER
I20260809 19:21:18.693863 19623 master.cc:552] Master@127.9.91.124:38053 shutdown complete.
I20260809 19:21:18.693903 19623 master_runner.cc:305] Clearing existing system tablet
I20260809 19:21:18.694020 19623 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.694054 19623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.694599 19476 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: attempting to promote NON_VOTER 508f35eb0fbb425b9c0f975d8ce0bcb4 to VOTER
I20260809 19:21:18.694691 19623 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.694770 19476 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } }
I20260809 19:21:18.695135 19778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.695163 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.695070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.0.0.1:36328)
0809 19:21:18.695104 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.695141 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 19:21:18.695243 19623 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.695286 19623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.695307 19623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
uuid: "da147739e9334c93922217ef735cd93a"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "c6557b13bd63e3d5930fceba1d9eaf24"
server_key_iv: "dc1894ca5818c64731b9d66d8f5185e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.695403 19551 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 LEARNER]: Refusing update from remote peer fbb92827f9434fa18ac3d334b7abcd44: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 19:21:18.695448 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.695381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.0.0.1:36332)
0809 19:21:18.695409 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.695430 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 19:21:18.695497 18582 consensus_peers.cc:487] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 -> Peer da147739e9334c93922217ef735cd93a (127.9.91.124:38053): Couldn't send request to peer da147739e9334c93922217ef735cd93a. Status: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38053: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:18.695571 19769 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 19:21:18.695972 19623 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:18.696101 19477 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 508f35eb0fbb425b9c0f975d8ce0bcb4 (127.9.91.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.696846 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.696913 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:18.696955 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.696991 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:18.697192 19623 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 19:21:18.697250 19623 log.cc:1196] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 19:21:18.697219 19551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 508f35eb0fbb425b9c0f975d8ce0bcb4 (127.9.91.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.697420 19623 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Deleting consensus metadata
I20260809 19:21:18.697576 19623 master_runner.cc:315] Copying system tablet from 127.9.91.126:34371
I20260809 19:21:18.698055 19772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.698139 19772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:21:18.698264 19623 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: tablet copy: Beginning tablet copy session from remote peer at address 127.9.91.126:34371
I20260809 19:21:18.705430 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.698532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.9.91.126:34371)
0809 19:21:18.698585 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.698587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.698768 (+ 181us) server_negotiation.cc:407] Connection header received
0809 19:21:18.698813 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.698814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.698823 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.698877 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.699339 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.700105 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.701194 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.701476 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.701540 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.702551 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.702553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.702553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.702559 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.704143 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.704615 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.704616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.704617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.704692 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.704907 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.704908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.704908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.705027 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.705071 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.705275 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.705351 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:21:18.705827 19788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.698581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:36596)
0809 19:21:18.698692 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.698702 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.698763 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.698881 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.698882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.699221 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.699261 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.699263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.700183 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.700184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.701111 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.701115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.701604 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.701606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.701671 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.702098 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.702102 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.702449 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.704252 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.704253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.704253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.704524 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.704771 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.704772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.704772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.704835 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.705120 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.705121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.705165 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.705637 (+ 472us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.705741 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":69,"threads_started":1}
I20260809 19:21:18.706131 18632 tablet_copy_service.cc:140] P fbb92827f9434fa18ac3d334b7abcd44: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer da147739e9334c93922217ef735cd93a ({username='oryx', principal='oryx/127.9.91.124@KRBTEST.COM'} at 127.0.0.1:36596)
I20260809 19:21:18.706185 18632 tablet_copy_service.cc:161] P fbb92827f9434fa18ac3d334b7abcd44: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer da147739e9334c93922217ef735cd93a at {username='oryx', principal='oryx/127.9.91.124@KRBTEST.COM'} at 127.0.0.1:36596: session id = da147739e9334c93922217ef735cd93a-00000000000000000000000000000000
I20260809 19:21:18.706715 18632 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 19:21:18.707201 19623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 19:21:18.708314 19623 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: tablet copy: Starting download of 0 data blocks...
I20260809 19:21:18.708437 19623 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: tablet copy: Starting download of 1 WAL segments...
I20260809 19:21:18.709069 19623 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 19:21:18.709843 18632 tablet_copy_service.cc:342] P fbb92827f9434fa18ac3d334b7abcd44: Request end of tablet copy session da147739e9334c93922217ef735cd93a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.91.124@KRBTEST.COM'} at 127.0.0.1:36596
I20260809 19:21:18.709890 18632 tablet_copy_service.cc:433] P fbb92827f9434fa18ac3d334b7abcd44: ending tablet copy session da147739e9334c93922217ef735cd93a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer da147739e9334c93922217ef735cd93a
I20260809 19:21:18.712209 19623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.713217 19789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.715804 19798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.715993 19623 server_base.cc:1034] running on GCE node
W20260809 19:21:18.720351 19792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.720556 19623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.720688 19623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.721797 19623 hybrid_clock.cc:648] HybridClock initialized: now 1786303278721789 us; error 26 us; skew 500 ppm
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.723202 19623 init.cc:377] Logged in from keytab as oryx/127.9.91.124@KRBTEST.COM (short username oryx)
I20260809 19:21:18.723474 19623 webserver.cc:460] Webserver started at http://127.9.91.124:36531/ using document root <none> and password file <none>
I20260809 19:21:18.723589 19623 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.723624 19623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.724309 19623 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:18.724874 19812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.725044 19623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.725093 19623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
uuid: "da147739e9334c93922217ef735cd93a"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "c6557b13bd63e3d5930fceba1d9eaf24"
server_key_iv: "dc1894ca5818c64731b9d66d8f5185e5"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.725140 19623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.763767 19623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.765518 19623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.769111 19623 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.124:38053
I20260809 19:21:18.769590 19623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
I20260809 19:21:18.769878 19891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.124:38053 every 8 connection(s)
I20260809 19:21:18.770388 19892 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:21:18.770848 19892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Bootstrap starting.
I20260809 19:21:18.773676 19892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:21:18.773866 19892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: Bootstrap complete.
I20260809 19:21:18.774050 19892 raft_consensus.cc:348] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } }
I20260809 19:21:18.774133 19892 raft_consensus.cc:729] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: da147739e9334c93922217ef735cd93a, State: Initialized, Role: LEARNER
I20260809 19:21:18.774168 19892 consensus_queue.cc:260] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } }
I20260809 19:21:18.774366 19892 sys_catalog.cc:564] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:18.774384 19897 sys_catalog.cc:455] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:18.774575 19897 sys_catalog.cc:458] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: This master's current role is: LEARNER
I20260809 19:21:18.776291 19914 catalog_manager.cc:1259] Loaded cluster ID: 61a364006954433e9f597857f71b3e8c
I20260809 19:21:18.776382 19914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: loading cluster ID for follower catalog manager: success
I20260809 19:21:18.777426 19914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: acquiring CA information for follower catalog manager: success
I20260809 19:21:18.777736 19914 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 19:21:18.781814 19770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:18.781994 19770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:18.782019 19770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:18.783193 19770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:18.783237 19770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:18.783265 19770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:18.784688 19770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab
--principal=oryx/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:42883
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=44027
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:18.784862 19770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:18.785043 19770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.789671 19922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.789678 19927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.789682 19921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.789989 19770 server_base.cc:1034] running on GCE node
I20260809 19:21:18.790184 19770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.790407 19770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.791529 19770 hybrid_clock.cc:648] HybridClock initialized: now 1786303278791516 us; error 31 us; skew 500 ppm
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.793764 19770 init.cc:377] Logged in from keytab as oryx/127.9.91.65@KRBTEST.COM (short username oryx)
I20260809 19:21:18.794130 19770 webserver.cc:460] Webserver started at http://127.9.91.65:44027/ using document root <none> and password file <none>
I20260809 19:21:18.794310 19770 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.794369 19770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.795657 19770 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:18.796135 19935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.796280 19770 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.796360 19770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "e6ee6bfb6754406e9e690f6428fa6fba"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "8a95963e9a42421a3791f8a067157e02"
server_key_iv: "b871b1493fe6eec2982f68ce06d30ec8"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.796638 19770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.819024 19770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.820806 19770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:18.820924 19770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.821117 19770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:18.821389 19770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:18.821429 19770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.821456 19770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:18.821480 19770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.829208 19770 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:42883
I20260809 19:21:18.829244 20106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:42883 every 8 connection(s)
I20260809 19:21:18.829531 19770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:18.833127 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 19770
I20260809 19:21:18.833304 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 18918
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.125@KRBTEST.COM
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.124@KRBTEST.COM
I20260809 19:21:18.852969 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:44323
--local_ip_for_outbound_sockets=127.9.91.66
--tserver_master_addrs=127.9.91.126:34371
--webserver_port=39277
--webserver_interface=127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab
--principal=oryx/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:21:18.853539 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.830309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:54551 (local address 127.9.91.125:38385)
0809 19:21:18.830357 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.830359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.830568 (+ 209us) server_negotiation.cc:407] Connection header received
0809 19:21:18.831145 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.831147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.831161 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.831192 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.832069 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.832815 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.833963 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.834346 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.834394 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.850240 (+ 15846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.850242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.850243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.850251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.851870 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.852347 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.852349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.852349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.852409 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.852691 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.852692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.852692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.852853 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.852897 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.853366 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.853449 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:21:18.853729 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.830285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:48017 (local address 127.9.91.126:34371)
0809 19:21:18.830334 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.830336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.830569 (+ 233us) server_negotiation.cc:407] Connection header received
0809 19:21:18.831119 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.831120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.831146 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.831192 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.832044 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.832807 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.833914 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.834149 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.834181 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.850240 (+ 16059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.850242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.850243 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.850251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.852577 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.852965 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.852966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.852966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.853023 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.853225 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.853226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.853226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.853350 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.853388 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.853570 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.853644 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:21:18.853895 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.830213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:56301 (local address 127.9.91.124:38053)
0809 19:21:18.830407 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.830408 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.830632 (+ 224us) server_negotiation.cc:407] Connection header received
0809 19:21:18.831163 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.831164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.831176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.831229 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.832052 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.832690 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.848868 (+ 16178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.849153 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.849208 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.851235 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.851237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.851237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.851245 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.852711 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.853086 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.853087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.853088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.853150 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.853315 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.853316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.853316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.853428 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.853479 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.853727 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.853811 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":48,"threads_started":1}
I20260809 19:21:18.854032 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.830187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.65:48017)
0809 19:21:18.830450 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.830492 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.831026 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.831514 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.831515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.831756 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.831951 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.831955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.832919 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.832920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.833828 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.833830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.834250 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.834251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.834405 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.847660 (+ 13255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.847670 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.850133 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.852658 (+ 2525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.852659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.852659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.852885 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.853090 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.853091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.853091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.853155 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.853433 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.853434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.853470 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.853794 (+ 324us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.853870 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":55,"thread_start_us":149,"threads_started":1}
I20260809 19:21:18.854171 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.830188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.9.91.65:56301)
0809 19:21:18.830544 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.830579 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.831030 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.831284 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.831287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.831765 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.831952 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.831958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.832796 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.832797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.848762 (+ 15965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.848765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.849265 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.849266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.849329 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.849723 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.849728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.851144 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.852795 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.852796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.852797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.853002 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.853209 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.853209 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.853210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.853261 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.853503 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.853503 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.853568 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.853630 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.854048 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":528,"thread_start_us":59,"threads_started":1}
I20260809 19:21:18.854318 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.830188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.9.91.65:54551)
0809 19:21:18.830458 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.830491 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.831024 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.831283 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.831287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.831762 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.831952 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.831958 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.832883 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.832884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.833844 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.833848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.834482 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.834483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.834561 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.847727 (+ 13166us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.847733 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.850128 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.851962 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.851970 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.851972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.852255 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.852480 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.852481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.852481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.852603 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.852965 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.852972 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.853218 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.853639 (+ 421us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.853800 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":58,"thread_start_us":180,"threads_started":1}
I20260809 19:21:18.855365 20107 heartbeater.cc:347] Connected to a master server at 127.9.91.124:38053
I20260809 19:21:18.855415 20109 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34371
I20260809 19:21:18.855367 20110 heartbeater.cc:347] Connected to a master server at 127.9.91.125:38385
I20260809 19:21:18.855542 20107 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.855568 20110 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.855736 20110 heartbeater.cc:511] Master 127.9.91.125:38385 requested a full tablet report, sending...
I20260809 19:21:18.855736 20107 heartbeater.cc:511] Master 127.9.91.124:38053 requested a full tablet report, sending...
I20260809 19:21:18.855542 20109 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.855985 20109 heartbeater.cc:511] Master 127.9.91.126:34371 requested a full tablet report, sending...
I20260809 19:21:18.856266 19839 ts_manager.cc:194] Registered new tserver with Master: e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65:42883)
I20260809 19:21:18.856266 19540 ts_manager.cc:194] Registered new tserver with Master: e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65:42883)
I20260809 19:21:18.856537 18604 ts_manager.cc:194] Re-registered known tserver with Master: e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65:42883)
I20260809 19:21:18.856894 18604 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:48017
W20260809 19:21:18.933480 20117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:18.933658 20117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:18.933679 20117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:18.934742 20117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:18.934777 20117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:18.934800 20117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:18.936239 20117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab
--principal=oryx/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:44323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=39277
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:18.936408 20117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:18.936575 20117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:18.941075 20125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.941071 20131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:18.941051 20126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:18.941301 20117 server_base.cc:1034] running on GCE node
I20260809 19:21:18.941490 20117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:18.941718 20117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:18.942852 20117 hybrid_clock.cc:648] HybridClock initialized: now 1786303278942839 us; error 30 us; skew 500 ppm
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:18.945214 20117 init.cc:377] Logged in from keytab as oryx/127.9.91.66@KRBTEST.COM (short username oryx)
I20260809 19:21:18.945576 20117 webserver.cc:460] Webserver started at http://127.9.91.66:39277/ using document root <none> and password file <none>
I20260809 19:21:18.945761 20117 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:18.945824 20117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:18.947022 20117 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:18.947535 20142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.947690 20117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:18.947772 20117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "0195befba7b040dfaed2d0055fc2405e"
format_stamp: "Formatted at 2026-08-09 19:21:17 on dist-test-slave-1vmg"
server_key: "fe0f3c45a700c6bdd8df70f3a9fca412"
server_key_iv: "a92d8ab19f40c03655d4c26259101730"
server_key_version: "encryptionkey@0"
I20260809 19:21:18.948040 20117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:18.966259 20117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:18.968194 20117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:18.968320 20117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:18.968513 20117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:18.968767 20117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:18.968799 20117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.968824 20117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:18.968849 20117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:18.974303 20117 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:44323
I20260809 19:21:18.974360 20333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:44323 every 8 connection(s)
I20260809 19:21:18.974753 20117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:18.978351 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 20117
I20260809 19:21:18.978498 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 19124
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.125@KRBTEST.COM
I20260809 19:21:18.985236 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.975411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:59751 (local address 127.9.91.125:38385)
0809 19:21:18.975706 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.975708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.976070 (+ 362us) server_negotiation.cc:407] Connection header received
0809 19:21:18.976664 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.976665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.976678 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.976708 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.977347 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.977829 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.979049 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.979279 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.979309 (+ 30us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.981473 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.981475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.981476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.981484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.983425 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.984043 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.984044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.984045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.984114 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.984467 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.984468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.984468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.984640 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.984693 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.985090 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.985161 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260809 19:21:18.985602 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.975386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.9.91.66:59751)
0809 19:21:18.976011 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.976018 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.976553 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.976780 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.976780 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.977081 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.977242 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.977248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.977925 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.977926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.978965 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.978968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.979376 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.979376 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.979483 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.979868 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.979875 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.981341 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.983532 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.983541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.983544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.983933 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.984200 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.984201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.984201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.984370 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.984753 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.984762 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.984961 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.985306 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.985439 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":9,"thread_start_us":91,"threads_started":1}
I20260809 19:21:18.987490 20337 heartbeater.cc:347] Connected to a master server at 127.9.91.125:38385
I20260809 19:21:18.987574 20337 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.987730 20337 heartbeater.cc:511] Master 127.9.91.125:38385 requested a full tablet report, sending...
I20260809 19:21:18.988131 19540 ts_manager.cc:194] Registered new tserver with Master: 0195befba7b040dfaed2d0055fc2405e (127.9.91.66:44323)
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.124@KRBTEST.COM
Aug 09 19:21:18 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303278, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:18.996978 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.975621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:33983 (local address 127.9.91.124:38053)
0809 19:21:18.975681 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.975684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.975940 (+ 256us) server_negotiation.cc:407] Connection header received
0809 19:21:18.976631 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.976633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.976652 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.976704 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.977329 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.978105 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.988787 (+ 10682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.989026 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.989057 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.990865 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.990866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.990867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.990874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.995738 (+ 4864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.996185 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.996186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.996187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.996254 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.996463 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.996464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.996465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.996590 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.996630 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.996810 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.996895 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:21:18.997303 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.975386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.9.91.66:33983)
0809 19:21:18.975878 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.975887 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.976550 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.976772 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.976775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.977069 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.977242 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.977246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.978181 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.978182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.988712 (+ 10530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.988714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.989113 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.989114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.989166 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.989507 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.989511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.990771 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.995827 (+ 5056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.995828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.995828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.996097 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.996322 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.996323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.996323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.996386 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.996673 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.996673 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.996713 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.997099 (+ 386us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.997210 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"spinlock_wait_cycles":6400,"thread_start_us":95,"threads_started":1}
I20260809 19:21:18.997614 20334 heartbeater.cc:347] Connected to a master server at 127.9.91.124:38053
I20260809 19:21:18.997668 20334 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:18.997725 20334 heartbeater.cc:511] Master 127.9.91.124:38053 requested a full tablet report, sending...
I20260809 19:21:18.998010 19839 ts_manager.cc:194] Registered new tserver with Master: 0195befba7b040dfaed2d0055fc2405e (127.9.91.66:44323)
I20260809 19:21:18.999778 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.975737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:32955 (local address 127.9.91.126:34371)
0809 19:21:18.975788 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:18.975790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:18.975800 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:18.976663 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:18.976665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:18.976678 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:18.976708 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:18.977356 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.978099 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.995174 (+ 17075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.995420 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.995484 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.997206 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:18.997207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:18.997208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:18.997215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:18.998696 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.999060 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.999061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.999062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.999116 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:18.999276 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:18.999276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:18.999277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:18.999380 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:18.999434 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:18.999611 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:21:18.999682 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:21:19.000056 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:18.975385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.66:32955)
0809 19:21:18.975552 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:18.975589 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:18.976538 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:18.976777 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:18.976778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:18.977078 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:18.977242 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.977248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.987604 (+ 10356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.987605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.988293 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:18.988295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:18.995494 (+ 7199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:18.995495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:18.995561 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:18.995913 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:18.995918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:18.997100 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:18.998764 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.998765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.998765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.998966 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.999167 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:18.999168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:18.999168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:18.999223 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:18.999461 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:18.999461 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:18.999501 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 19:21:18.999563 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:21:18.999655 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260809 19:21:19.000437 20335 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34371
I20260809 19:21:19.000488 20335 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:19.000553 20335 heartbeater.cc:511] Master 127.9.91.126:34371 requested a full tablet report, sending...
I20260809 19:21:19.000536 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:41079
--local_ip_for_outbound_sockets=127.9.91.67
--tserver_master_addrs=127.9.91.126:34371
--webserver_port=38333
--webserver_interface=127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab
--principal=oryx/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:21:19.000799 18604 ts_manager.cc:194] Re-registered known tserver with Master: 0195befba7b040dfaed2d0055fc2405e (127.9.91.66:44323)
I20260809 19:21:19.001144 18604 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:32955
W20260809 19:21:19.079859 20345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:19.080025 20345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:19.080047 20345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:19.081104 20345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:19.081144 20345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:19.081168 20345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:19.082522 20345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab
--principal=oryx/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:41079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=38333
--tserver_master_addrs=127.9.91.126:34371,127.9.91.125:38385,127.9.91.124:38053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:19.082681 20345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:19.082834 20345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:19.087370 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:19.087359 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:19.087360 20358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:19.087616 20345 server_base.cc:1034] running on GCE node
I20260809 19:21:19.087807 20345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:19.088018 20345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:19.089159 20345 hybrid_clock.cc:648] HybridClock initialized: now 1786303279089147 us; error 30 us; skew 500 ppm
Aug 09 19:21:19 dist-test-slave-1vmg krb5kdc[18536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303279, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:19.091460 20345 init.cc:377] Logged in from keytab as oryx/127.9.91.67@KRBTEST.COM (short username oryx)
I20260809 19:21:19.091868 20345 webserver.cc:460] Webserver started at http://127.9.91.67:38333/ using document root <none> and password file <none>
I20260809 19:21:19.092047 20345 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:19.092104 20345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:19.093283 20345 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:19.093792 20367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:19.093943 20345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:19.094019 20345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "d1bba31429c34198b9b265c107d9314d"
format_stamp: "Formatted at 2026-08-09 19:21:18 on dist-test-slave-1vmg"
server_key: "eb3c1e4b0aac61bbdfa8a1f635080862"
server_key_iv: "c97adb73ca411f71d4f4eba54e94e8f7"
server_key_version: "encryptionkey@0"
I20260809 19:21:19.094288 20345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:19.114110 20345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:19.115850 20345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:19.115976 20345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:19.116165 20345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:19.116415 20345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:19.116446 20345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:19.116472 20345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:19.116496 20345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:19.122394 20345 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:41079
I20260809 19:21:19.122437 20541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:41079 every 8 connection(s)
I20260809 19:21:19.122722 20345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:19.125397 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 20345
Aug 09 19:21:19 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303279, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.125@KRBTEST.COM
Aug 09 19:21:19 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303279, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.124@KRBTEST.COM
Aug 09 19:21:19 dist-test-slave-1vmg krb5kdc[18536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303279, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:19.135552 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.127439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.9.91.124:38053)
0809 19:21:19.128348 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.128349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.128687 (+ 338us) server_negotiation.cc:407] Connection header received
0809 19:21:19.128710 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.128711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.128723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.128791 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.129481 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.130203 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.131254 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.131533 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.131596 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.132886 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.132887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.132888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.132894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.134413 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.134776 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.134777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.134777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.134831 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.135060 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.135061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.135061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.135184 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.135279 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.135387 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.135460 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":65,"threads_started":1}
I20260809 19:21:19.136176 20559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.126444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.9.91.126:34371)
0809 19:21:19.126743 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.126744 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.126888 (+ 144us) server_negotiation.cc:407] Connection header received
0809 19:21:19.126907 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.126907 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.126939 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.126992 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.127744 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.128432 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.129408 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.129695 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.129752 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.131168 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.131170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.131170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.131177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.133238 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.134518 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.134519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.134519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.134793 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.135035 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.135036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.135036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.135427 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.135512 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.135996 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.136072 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":73,"threads_started":1}
I20260809 19:21:19.136572 20548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.123415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.67:36187)
0809 19:21:19.123972 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.123983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.124130 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.124473 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.124474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.124762 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:19.124955 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.124960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.125932 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.125933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.127219 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.127222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.127753 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.127754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.127883 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.128344 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:19.128353 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:19.131753 (+ 3400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:19.133968 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:19.133976 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:19.133978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:19.134432 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:19.135597 (+ 1165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:19.135598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:19.135598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:19.135714 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:19.135960 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:19.135967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:19.136125 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.136219 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.136370 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":79,"thread_start_us":70,"threads_started":1}
I20260809 19:21:19.136703 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.123343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.9.91.67:43091)
0809 19:21:19.123551 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.123591 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.124116 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.124339 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.124342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.124760 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:19.124955 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.124960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.126338 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.126339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.127225 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.127228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.128089 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.128090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.128194 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.128563 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:19.128568 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:19.132281 (+ 3713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:19.135195 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:19.135196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:19.135196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:19.135381 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:19.136116 (+ 735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:19.136117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:19.136117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:19.136171 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:19.136432 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:19.136433 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:19.136471 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.136540 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.136623 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":10,"thread_start_us":82,"threads_started":1}
I20260809 19:21:19.136870 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.123430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:36187 (local address 127.9.91.126:34371)
0809 19:21:19.123715 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.123717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.124037 (+ 320us) server_negotiation.cc:407] Connection header received
0809 19:21:19.124336 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.124338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.124349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.124411 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.125064 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.125756 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.127322 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.127624 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.127664 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.131848 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.131850 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.131850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.131857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.133878 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.135457 (+ 1579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.135458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.135458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.135526 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.135785 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.135786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.135786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.135905 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.135965 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.136715 (+ 750us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.136792 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260809 19:21:19.137040 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.123616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:43091 (local address 127.9.91.124:38053)
0809 19:21:19.123679 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.123681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.123689 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:19.124216 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.124218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.124231 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.124280 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.125071 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.125828 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.127702 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.127975 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.128015 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.132589 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.132590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.132591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.132598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.135121 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.135994 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.135995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.135995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.136050 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.136237 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.136238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.136238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.136348 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.136406 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.136903 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.136980 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:21:19.137313 20543 heartbeater.cc:347] Connected to a master server at 127.9.91.124:38053
I20260809 19:21:19.137394 20543 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:19.137540 20545 heartbeater.cc:347] Connected to a master server at 127.9.91.126:34371
I20260809 19:21:19.137557 20543 heartbeater.cc:511] Master 127.9.91.124:38053 requested a full tablet report, sending...
I20260809 19:21:19.137581 20545 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:19.137627 20545 heartbeater.cc:511] Master 127.9.91.126:34371 requested a full tablet report, sending...
I20260809 19:21:19.137864 18604 ts_manager.cc:194] Re-registered known tserver with Master: d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079)
I20260809 19:21:19.137908 19839 ts_manager.cc:194] Registered new tserver with Master: d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079)
I20260809 19:21:19.138293 18604 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:36187
I20260809 19:21:19.140046 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.129769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.9.91.125:38385)
0809 19:21:19.130118 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.130120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.130207 (+ 87us) server_negotiation.cc:407] Connection header received
0809 19:21:19.130239 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.130240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.130251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.130303 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.131038 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.131793 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.132742 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.133035 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.133093 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.134313 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.134314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.134314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.134321 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.137830 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.139251 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.139252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.139252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.139307 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.139490 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.139490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.139491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.139615 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.139706 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.139839 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.139917 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":127,"threads_started":1}
I20260809 19:21:19.138521 18604 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 19:21:19.140128 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.123906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:41459 (local address 127.9.91.125:38385)
0809 19:21:19.123966 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.123968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.124233 (+ 265us) server_negotiation.cc:407] Connection header received
0809 19:21:19.124324 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.124326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.124338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.124386 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.125041 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.125836 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.127322 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.127624 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.127664 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.137388 (+ 9724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.137389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.137389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.137396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.138892 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.139273 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.139274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.139275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.139338 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.139534 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.139535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.139536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.139643 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.139699 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.139904 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.139972 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 19:21:19.140424 18604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:19.140439 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.123830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.9.91.67:41459)
0809 19:21:19.124175 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.124184 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.124268 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.124396 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.124397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.124732 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:19.124954 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.124959 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.125978 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.125979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.127221 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.127222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.127751 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.127752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.127883 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.128602 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:19.128607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:19.137299 (+ 8692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:19.138989 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:19.138990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:19.138990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:19.139188 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:19.139397 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:19.139398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:19.139398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:19.139451 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:19.139726 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:19.139727 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:19.139790 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.140152 (+ 362us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.140249 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":80,"thread_start_us":63,"threads_started":1}
I20260809 19:21:19.140866 20546 heartbeater.cc:347] Connected to a master server at 127.9.91.125:38385
I20260809 19:21:19.140915 20546 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:19.140980 20546 heartbeater.cc:511] Master 127.9.91.125:38385 requested a full tablet report, sending...
I20260809 19:21:19.141227 19540 ts_manager.cc:194] Registered new tserver with Master: d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079)
I20260809 19:21:19.147647 20575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.143765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:41079 (local address 127.0.0.1:43922)
0809 19:21:19.143892 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.143901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.143944 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.144610 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.144611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.144612 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:19.144653 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.144654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.145833 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.145834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.146723 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.146725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.147413 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.147414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.147474 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.147477 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.147530 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.147569 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260809 19:21:19.147831 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.143691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44323 (local address 127.0.0.1:52380)
0809 19:21:19.143732 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.143741 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.143774 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.144296 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.144298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.144300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:19.144335 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.144337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.145219 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.145220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.145956 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.145958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.147637 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.147639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.147698 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.147701 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.147729 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.147766 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:21:19.147843 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.144010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.9.91.67:41079)
0809 19:21:19.144389 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.144393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.144402 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:19.144508 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.144510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.144523 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.144719 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:19.144757 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.145750 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.146791 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.147338 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.147401 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.147468 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.147545 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.147575 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":83,"threads_started":1}
I20260809 19:21:19.147956 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.143933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42883 (local address 127.0.0.1:37568)
0809 19:21:19.144036 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.144045 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.144086 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.144567 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.144567 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.144570 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:19.144606 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.144608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.146087 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.146088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.146868 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.146870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.147778 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.147778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.147834 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.147837 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.147865 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.147899 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":68,"threads_started":1}
I20260809 19:21:19.148308 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.144119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.9.91.65:42883)
0809 19:21:19.144400 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.144404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.144414 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:19.144453 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.144455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.144469 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.144681 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:19.144751 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.146017 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.146984 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.147690 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.148064 (+ 374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.148159 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.148186 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.148226 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260809 19:21:19.148415 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.143799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.9.91.66:44323)
0809 19:21:19.144136 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.144141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.144150 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:19.144191 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.144193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.144206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.144284 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:19.144422 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.145144 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.146812 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.147526 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.148209 (+ 683us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.148284 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.148307 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.148343 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260809 19:21:19.148514 20441 tablet_service.cc:1467] Processing CreateTablet for tablet 83f08f70fe1242109ab3075b17ad5a28 (DEFAULT_TABLE table=test-table [id=8be32314ba9541109326979826e07ada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:19.148838 20441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83f08f70fe1242109ab3075b17ad5a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:19.148938 20225 tablet_service.cc:1467] Processing CreateTablet for tablet 83f08f70fe1242109ab3075b17ad5a28 (DEFAULT_TABLE table=test-table [id=8be32314ba9541109326979826e07ada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:19.148973 20007 tablet_service.cc:1467] Processing CreateTablet for tablet 83f08f70fe1242109ab3075b17ad5a28 (DEFAULT_TABLE table=test-table [id=8be32314ba9541109326979826e07ada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:19.149174 20225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83f08f70fe1242109ab3075b17ad5a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:19.149302 20007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83f08f70fe1242109ab3075b17ad5a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:19.152061 20585 tablet_bootstrap.cc:492] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Bootstrap starting.
I20260809 19:21:19.152164 20584 tablet_bootstrap.cc:492] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Bootstrap starting.
I20260809 19:21:19.152612 20585 tablet_bootstrap.cc:654] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:19.152793 20584 tablet_bootstrap.cc:654] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:19.152916 20585 log.cc:824] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:19.152930 20586 tablet_bootstrap.cc:492] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Bootstrap starting.
I20260809 19:21:19.153141 20584 log.cc:824] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:19.153517 20586 tablet_bootstrap.cc:654] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:19.153846 20586 log.cc:824] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:19.153954 20584 tablet_bootstrap.cc:492] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: No bootstrap required, opened a new log
I20260809 19:21:19.154003 20584 ts_tablet_manager.cc:1397] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:21:19.154036 20585 tablet_bootstrap.cc:492] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: No bootstrap required, opened a new log
I20260809 19:21:19.154090 20585 ts_tablet_manager.cc:1397] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:19.155570 20586 tablet_bootstrap.cc:492] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: No bootstrap required, opened a new log
I20260809 19:21:19.155515 20585 raft_consensus.cc:348] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.155622 20585 raft_consensus.cc:374] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:19.155640 20586 ts_tablet_manager.cc:1397] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:21:19.155650 20585 raft_consensus.cc:729] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bba31429c34198b9b265c107d9314d, State: Initialized, Role: FOLLOWER
I20260809 19:21:19.155776 20585 consensus_queue.cc:260] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.156054 20585 ts_tablet_manager.cc:1428] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:21:19.156154 20545 heartbeater.cc:503] Master 127.9.91.126:34371 was elected leader, sending a full tablet report...
I20260809 19:21:19.156149 20584 raft_consensus.cc:348] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.156301 20584 raft_consensus.cc:374] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:19.156332 20584 raft_consensus.cc:729] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0195befba7b040dfaed2d0055fc2405e, State: Initialized, Role: FOLLOWER
I20260809 19:21:19.156448 20584 consensus_queue.cc:260] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.156683 20584 ts_tablet_manager.cc:1428] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:21:19.156845 20335 heartbeater.cc:503] Master 127.9.91.126:34371 was elected leader, sending a full tablet report...
I20260809 19:21:19.157480 20586 raft_consensus.cc:348] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.157639 20586 raft_consensus.cc:374] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:19.157675 20586 raft_consensus.cc:729] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ee6bfb6754406e9e690f6428fa6fba, State: Initialized, Role: FOLLOWER
I20260809 19:21:19.157960 20586 consensus_queue.cc:260] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.158219 20586 ts_tablet_manager.cc:1428] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:21:19.158385 20109 heartbeater.cc:503] Master 127.9.91.126:34371 was elected leader, sending a full tablet report...
W20260809 19:21:19.225909 20339 tablet.cc:2369] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:21:19.276922 20594 raft_consensus.cc:482] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:19.277025 20594 raft_consensus.cc:504] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.277334 20594 leader_election.cc:288] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0195befba7b040dfaed2d0055fc2405e (127.9.91.66:44323), d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079)
I20260809 19:21:19.280022 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.277568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:42739 (local address 127.9.91.67:41079)
0809 19:21:19.277614 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.277616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.277625 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:19.277692 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.277694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.277705 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.277750 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:19.277882 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.278346 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.279356 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.279718 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.279786 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.279796 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.279920 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.279959 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:21:19.280036 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.277474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:41079 (local address 127.9.91.65:42739)
0809 19:21:19.277569 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.277580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.277641 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.277756 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.277757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.277758 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:19.277830 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.277831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.278414 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.278415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.279268 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.279271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.279793 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.279794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.279863 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.279866 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.279910 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.279953 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":63}
I20260809 19:21:19.280028 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.277529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:33115 (local address 127.9.91.66:44323)
0809 19:21:19.277601 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.277603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.277663 (+ 60us) server_negotiation.cc:407] Connection header received
0809 19:21:19.277725 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.277726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.277737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.277780 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:19.277909 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.278364 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.279349 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.279718 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.279789 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.279799 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.279929 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.279964 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 19:21:19.280061 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.277482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44323 (local address 127.9.91.65:33115)
0809 19:21:19.277615 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.277624 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.277673 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.277788 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.277789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.277792 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:19.277852 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.277853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.278433 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.278434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.279268 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.279271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.279797 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.279798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.279870 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.279873 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.279918 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.279962 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":39}
I20260809 19:21:19.280360 20268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f08f70fe1242109ab3075b17ad5a28" candidate_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0195befba7b040dfaed2d0055fc2405e" is_pre_election: true
I20260809 19:21:19.280484 20268 raft_consensus.cc:2454] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ee6bfb6754406e9e690f6428fa6fba in term 0.
I20260809 19:21:19.280786 19937 leader_election.cc:302] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0195befba7b040dfaed2d0055fc2405e, e6ee6bfb6754406e9e690f6428fa6fba; no voters:
I20260809 19:21:19.280815 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f08f70fe1242109ab3075b17ad5a28" candidate_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bba31429c34198b9b265c107d9314d" is_pre_election: true
I20260809 19:21:19.280907 20594 raft_consensus.cc:2781] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:19.280939 20472 raft_consensus.cc:2454] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ee6bfb6754406e9e690f6428fa6fba in term 0.
I20260809 19:21:19.280969 20594 raft_consensus.cc:482] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:19.280999 20594 raft_consensus.cc:3037] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:19.281723 20594 raft_consensus.cc:504] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.281834 20594 leader_election.cc:288] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [CANDIDATE]: Term 1 election: Requested vote from peers 0195befba7b040dfaed2d0055fc2405e (127.9.91.66:44323), d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079)
I20260809 19:21:19.281996 20268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f08f70fe1242109ab3075b17ad5a28" candidate_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0195befba7b040dfaed2d0055fc2405e"
I20260809 19:21:19.282003 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83f08f70fe1242109ab3075b17ad5a28" candidate_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bba31429c34198b9b265c107d9314d"
I20260809 19:21:19.282084 20268 raft_consensus.cc:3037] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:19.282087 20472 raft_consensus.cc:3037] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:19.282964 20268 raft_consensus.cc:2454] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ee6bfb6754406e9e690f6428fa6fba in term 1.
I20260809 19:21:19.283108 19937 leader_election.cc:302] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0195befba7b040dfaed2d0055fc2405e, e6ee6bfb6754406e9e690f6428fa6fba; no voters:
I20260809 19:21:19.283245 20594 raft_consensus.cc:2781] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:19.283314 20472 raft_consensus.cc:2454] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ee6bfb6754406e9e690f6428fa6fba in term 1.
I20260809 19:21:19.283398 20594 raft_consensus.cc:686] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 LEADER]: Becoming Leader. State: Replica: e6ee6bfb6754406e9e690f6428fa6fba, State: Running, Role: LEADER
I20260809 19:21:19.283557 20594 consensus_queue.cc:237] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:19.284392 18604 catalog_manager.cc:5495] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba reported cstate change: term changed from 0 to 1, leader changed from <none> to e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65). New cstate: current_term: 1 leader_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } health_report { overall_health: HEALTHY } } }
I20260809 19:21:19.289834 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.287458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.0.0.1:36354)
0809 19:21:19.287548 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:19.287557 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:19.287600 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:19.287718 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:19.287719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:19.287719 (+ 0us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:19.287753 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.287755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.288271 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.288271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.288989 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:19.288992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.289603 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.289605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:19.289669 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.289672 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:19.289727 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:21:19.289768 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 19:21:19.289831 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.287509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.9.91.124:38053)
0809 19:21:19.287565 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.287567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.287588 (+ 21us) server_negotiation.cc:407] Connection header received
0809 19:21:19.287649 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.287650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.287668 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.287717 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:19.287786 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.288223 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.289072 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.289544 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.289601 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.289679 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.289733 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.289771 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:21:19.290198 19857 raft_consensus.cc:1203] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.10] Dedup: 1.7->[1.8-1.10]
I20260809 19:21:19.291180 20598 sys_catalog.cc:455] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbb92827f9434fa18ac3d334b7abcd44. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:19.291270 20598 sys_catalog.cc:458] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: This master's current role is: LEARNER
I20260809 19:21:19.292905 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.290452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.9.91.65:42883)
0809 19:21:19.290487 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.290488 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.290529 (+ 41us) server_negotiation.cc:407] Connection header received
0809 19:21:19.290604 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.290604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.290618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.290676 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:19.290834 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.291523 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.292364 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.292545 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.292598 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.292702 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:19.292782 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:19.292831 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.292845 (+ 14us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.292859 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:19.294322 20472 raft_consensus.cc:1261] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Refusing update from remote peer e6ee6bfb6754406e9e690f6428fa6fba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:19.294346 20268 raft_consensus.cc:1261] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 1 FOLLOWER]: Refusing update from remote peer e6ee6bfb6754406e9e690f6428fa6fba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:19.294616 20594 consensus_queue.cc:1035] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:19.294730 20595 consensus_queue.cc:1035] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:19.297472 20609 mvcc.cc:205] Tried to move back new op lower bound from 7316698231987306496 to 7316698231945924608. Current Snapshot: MvccSnapshot[applied={T|T < 7316698231987306496}]
I20260809 19:21:19.298843 20610 mvcc.cc:205] Tried to move back new op lower bound from 7316698231987306496 to 7316698231945924608. Current Snapshot: MvccSnapshot[applied={T|T < 7316698231987306496}]
I20260809 19:21:19.299535 20608 mvcc.cc:205] Tried to move back new op lower bound from 7316698231987306496 to 7316698231945924608. Current Snapshot: MvccSnapshot[applied={T|T < 7316698231987306496 or (T in {7316698231987306496})}]
I20260809 19:21:19.300849 18604 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:19.300942 18604 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:19.301352 19476 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: NON_VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: true } } }
I20260809 19:21:19.301437 19476 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:19.302296 19769 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: attempting to promote NON_VOTER da147739e9334c93922217ef735cd93a to VOTER
I20260809 19:21:19.302413 19769 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } }
I20260809 19:21:19.302968 19850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 LEARNER]: Refusing update from remote peer fbb92827f9434fa18ac3d334b7abcd44: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260809 19:21:19.302989 19551 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Refusing update from remote peer fbb92827f9434fa18ac3d334b7abcd44: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260809 19:21:19.303254 19477 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260809 19:21:19.303334 18604 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44: Sending DeleteTablet for 3 replicas of tablet 83f08f70fe1242109ab3075b17ad5a28
I20260809 19:21:19.303637 20441 tablet_service.cc:1514] Processing DeleteTablet for tablet 83f08f70fe1242109ab3075b17ad5a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:19 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:43922
I20260809 19:21:19.303644 20225 tablet_service.cc:1514] Processing DeleteTablet for tablet 83f08f70fe1242109ab3075b17ad5a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:19 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:52380
I20260809 19:21:19.303905 20007 tablet_service.cc:1514] Processing DeleteTablet for tablet 83f08f70fe1242109ab3075b17ad5a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:19 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:37568
I20260809 19:21:19.304226 20619 tablet_replica.cc:330] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: stopping tablet replica
I20260809 19:21:19.304319 20619 raft_consensus.cc:2229] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:19.304414 20619 raft_consensus.cc:2258] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:19.304109 20624 tablet_replica.cc:330] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: stopping tablet replica
I20260809 19:21:19.304594 20624 raft_consensus.cc:2229] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:19.304718 20624 raft_consensus.cc:2258] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:19.304912 19477 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [term 1 LEADER]: Committing config change with OpId 1.12: config changed from index 7 to 12, da147739e9334c93922217ef735cd93a (127.9.91.124) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:19.305191 20619 ts_tablet_manager.cc:1905] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:19.305775 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:19.305850 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:19.305898 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:19.305938 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:19.306041 19476 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fbb92827f9434fa18ac3d334b7abcd44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260809 19:21:19.306542 19551 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 7 to 12, da147739e9334c93922217ef735cd93a (127.9.91.124) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:19.306496 19850 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 7 to 12, da147739e9334c93922217ef735cd93a (127.9.91.124) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:19.307478 20598 sys_catalog.cc:455] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:19.307562 20598 sys_catalog.cc:458] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:21:19.308090 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 18558
I20260809 19:21:19.313280 20619 ts_tablet_manager.cc:1918] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:19.313375 20619 log.cc:1196] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/83f08f70fe1242109ab3075b17ad5a28
I20260809 19:21:19.313741 20619 ts_tablet_manager.cc:1939] T 83f08f70fe1242109ab3075b17ad5a28 P 0195befba7b040dfaed2d0055fc2405e: Deleting consensus metadata
I20260809 19:21:19.314248 20628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fbb92827f9434fa18ac3d334b7abcd44" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:19.314359 20628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:21:19.317533 20621 tablet_replica.cc:330] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: stopping tablet replica
I20260809 19:21:19.317639 20621 raft_consensus.cc:2229] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:19.317744 20621 raft_consensus.cc:2258] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:19.317749 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.304103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.9.91.124:38053)
0809 19:21:19.304138 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.304140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.304205 (+ 65us) server_negotiation.cc:407] Connection header received
0809 19:21:19.304350 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.304351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.304362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.304389 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.305631 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.306350 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.307641 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.307945 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.307988 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.315032 (+ 7044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.315033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.315034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.315041 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.316524 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.316956 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.316957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.316957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.317026 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.317254 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.317254 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.317255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.317353 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.317435 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.317594 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.317672 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:19.318284 20621 ts_tablet_manager.cc:1905] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:19.319247 20624 ts_tablet_manager.cc:1905] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:19.319823 20621 ts_tablet_manager.cc:1918] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:19.319902 20621 log.cc:1196] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/83f08f70fe1242109ab3075b17ad5a28
I20260809 19:21:19.320173 20621 ts_tablet_manager.cc:1939] T 83f08f70fe1242109ab3075b17ad5a28 P d1bba31429c34198b9b265c107d9314d: Deleting consensus metadata
I20260809 19:21:19.320957 20624 ts_tablet_manager.cc:1918] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:19.321050 20624 log.cc:1196] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/83f08f70fe1242109ab3075b17ad5a28
I20260809 19:21:19.321257 20624 ts_tablet_manager.cc:1939] T 83f08f70fe1242109ab3075b17ad5a28 P e6ee6bfb6754406e9e690f6428fa6fba: Deleting consensus metadata
W20260809 19:21:19.325574 20149 connection.cc:535] server connection from 127.0.0.1:52380 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:21:19.325601 20370 connection.cc:535] server connection from 127.0.0.1:43922 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:21:19.332315 19937 connection.cc:535] server connection from 127.0.0.1:37568 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:21:19.334838 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.327541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.9.91.125:38385)
0809 19:21:19.327579 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.327581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.327775 (+ 194us) server_negotiation.cc:407] Connection header received
0809 19:21:19.327863 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.327865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.327874 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.327902 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.328560 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.329297 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.330264 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.330555 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.330590 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.332358 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.332359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.332360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.332367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.333804 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.334137 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.334138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.334138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.334195 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.334356 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.334357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.334357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.334468 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.334551 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.334683 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.334755 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:21:19.335724 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:19.327635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36370 (local address 127.9.91.124:38053)
0809 19:21:19.327668 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:19.327669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:19.327676 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:19.327787 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:19.327788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:19.327797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:19.327826 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:19.328761 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.329666 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.330935 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:19.331305 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:19.331374 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:19.332922 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:19.332923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:19.332924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:19.332931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:19.334540 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.334869 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.334870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.334870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.334948 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:19.335165 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:19.335166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:19.335167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:19.335294 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:19.335360 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:19.335563 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 19:21:19.335641 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:20.298943 20637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.298637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.66:48211)
0809 19:21:20.298759 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.298858 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260809 19:21:20.299073 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.298724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.65:32821)
0809 19:21:20.298863 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.298956 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
W20260809 19:21:20.299206 20335 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:34371 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
W20260809 19:21:20.299244 20109 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:34371 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
I20260809 19:21:20.299957 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.299725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.67:40223)
0809 19:21:20.299845 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.299913 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260809 19:21:20.300163 20545 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:34371 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
I20260809 19:21:20.806658 20641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fbb92827f9434fa18ac3d334b7abcd44)
I20260809 19:21:20.806820 20641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } }
I20260809 19:21:20.806941 20643 raft_consensus.cc:482] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fbb92827f9434fa18ac3d334b7abcd44)
I20260809 19:21:20.807070 20643 raft_consensus.cc:504] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } }
I20260809 19:21:20.807144 20641 leader_election.cc:288] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbb92827f9434fa18ac3d334b7abcd44 (127.9.91.126:34371), da147739e9334c93922217ef735cd93a (127.9.91.124:38053)
I20260809 19:21:20.807545 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.807232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37126)
0809 19:21:20.807431 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.807486 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":56,"threads_started":1}
I20260809 19:21:20.807668 20643 leader_election.cc:288] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbb92827f9434fa18ac3d334b7abcd44 (127.9.91.126:34371), 508f35eb0fbb425b9c0f975d8ce0bcb4 (127.9.91.125:38385)
W20260809 19:21:20.807683 19516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
I20260809 19:21:20.807767 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.807412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37142)
0809 19:21:20.807673 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.807728 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":57,"threads_started":1}
W20260809 19:21:20.807956 19818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
I20260809 19:21:20.808341 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.808240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37156)
0809 19:21:20.808285 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.808310 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:21:20.808444 20650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.808341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37168)
0809 19:21:20.808386 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.808416 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:21:20.808444 19516 leader_election.cc:334] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fbb92827f9434fa18ac3d334b7abcd44 (127.9.91.126:34371): Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
W20260809 19:21:20.808568 19818 leader_election.cc:334] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fbb92827f9434fa18ac3d334b7abcd44 (127.9.91.126:34371): Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
I20260809 19:21:20.810366 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.807479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36374 (local address 127.9.91.124:38053)
0809 19:21:20.807775 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:20.807778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:20.807786 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:20.807812 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:20.807813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:20.807824 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:20.807920 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:20.808052 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.808763 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.809697 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.810073 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.810132 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.810145 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:20.810265 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 19:21:20.810303 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":45,"threads_started":1}
I20260809 19:21:20.810370 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.807283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38053 (local address 127.0.0.1:36374)
0809 19:21:20.807479 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.807490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:20.807541 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:20.807929 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:20.807930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:20.807932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:20.807991 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.807994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.808827 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.808828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.809617 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.809620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.810148 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.810149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.810208 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.810211 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:20.810256 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:20.810301 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":2,"thread_start_us":59,"threads_started":1}
I20260809 19:21:20.810590 19850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "da147739e9334c93922217ef735cd93a" is_pre_election: true
I20260809 19:21:20.810683 19850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508f35eb0fbb425b9c0f975d8ce0bcb4 in term 1.
I20260809 19:21:20.810822 19514 leader_election.cc:302] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508f35eb0fbb425b9c0f975d8ce0bcb4, da147739e9334c93922217ef735cd93a; no voters: fbb92827f9434fa18ac3d334b7abcd44
I20260809 19:21:20.810959 20641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 19:21:20.810994 20641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fbb92827f9434fa18ac3d334b7abcd44)
I20260809 19:21:20.811014 20641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:21:20.811184 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.807409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38385 (local address 127.0.0.1:55368)
0809 19:21:20.807531 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.807541 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:20.807600 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:20.808711 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:20.808712 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:20.808715 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:20.808770 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.808772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.809462 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.809463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.810226 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.810228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.810960 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.810961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.811027 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.811031 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:20.811090 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:21:20.811134 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":41,"threads_started":1}
I20260809 19:21:20.811190 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.807584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55368 (local address 127.9.91.125:38385)
0809 19:21:20.807794 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:20.807797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:20.807887 (+ 90us) server_negotiation.cc:407] Connection header received
0809 19:21:20.808614 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:20.808615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:20.808627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:20.808701 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:20.808846 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.809380 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.810315 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.810856 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.810953 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.810967 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:20.811094 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 19:21:20.811137 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":195,"thread_start_us":59,"threads_started":1}
I20260809 19:21:20.811384 19551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da147739e9334c93922217ef735cd93a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" is_pre_election: true
I20260809 19:21:20.811866 20641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } }
I20260809 19:21:20.812002 19551 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da147739e9334c93922217ef735cd93a in current term 2: Already voted for candidate 508f35eb0fbb425b9c0f975d8ce0bcb4 in this term.
I20260809 19:21:20.812001 20641 leader_election.cc:288] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [CANDIDATE]: Term 2 election: Requested vote from peers fbb92827f9434fa18ac3d334b7abcd44 (127.9.91.126:34371), da147739e9334c93922217ef735cd93a (127.9.91.124:38053)
I20260809 19:21:20.812188 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.812090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37176)
0809 19:21:20.812134 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.812160 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:21:20.812166 19850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "da147739e9334c93922217ef735cd93a"
I20260809 19:21:20.812198 19813 leader_election.cc:302] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da147739e9334c93922217ef735cd93a; no voters: 508f35eb0fbb425b9c0f975d8ce0bcb4, fbb92827f9434fa18ac3d334b7abcd44
I20260809 19:21:20.812228 19850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:21:20.812510 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.812414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37188)
0809 19:21:20.812460 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.812490 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:21:20.812582 19516 leader_election.cc:334] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fbb92827f9434fa18ac3d334b7abcd44 (127.9.91.126:34371): Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
I20260809 19:21:20.812947 19850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508f35eb0fbb425b9c0f975d8ce0bcb4 in term 2.
I20260809 19:21:20.813025 20643 raft_consensus.cc:2726] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:21:20.813095 19514 leader_election.cc:302] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508f35eb0fbb425b9c0f975d8ce0bcb4, da147739e9334c93922217ef735cd93a; no voters: fbb92827f9434fa18ac3d334b7abcd44
I20260809 19:21:20.813197 20641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:21:20.813330 20641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [term 2 LEADER]: Becoming Leader. State: Replica: 508f35eb0fbb425b9c0f975d8ce0bcb4, State: Running, Role: LEADER
I20260809 19:21:20.813402 20641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } }
I20260809 19:21:20.813699 20660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 508f35eb0fbb425b9c0f975d8ce0bcb4. Latest consensus state: current_term: 2 leader_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:20.813798 20660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:20.813966 20664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:20.814332 20664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:20.814400 20664 catalog_manager.cc:1259] Loaded cluster ID: 61a364006954433e9f597857f71b3e8c
I20260809 19:21:20.814432 20664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:20.814576 20664 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:20.814652 20664 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Loaded TSK: 0
I20260809 19:21:20.814785 20664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:20.858338 19540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:20.859908 19540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:20.860790 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.860706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37196)
0809 19:21:20.860740 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.860770 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 19:21:20.860798 19850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [term 2 FOLLOWER]: Refusing update from remote peer 508f35eb0fbb425b9c0f975d8ce0bcb4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 19:21:20.861086 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.860998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.0.0.1:37210)
0809 19:21:20.861042 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.861067 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:21:20.861099 20660 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 19:21:20.861214 19516 consensus_peers.cc:487] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 -> Peer fbb92827f9434fa18ac3d334b7abcd44 (127.9.91.126:34371): Couldn't send request to peer fbb92827f9434fa18ac3d334b7abcd44. Status: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 19:21:20.861683 20643 sys_catalog.cc:455] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 508f35eb0fbb425b9c0f975d8ce0bcb4. Latest consensus state: current_term: 2 leader_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:20.861778 20643 sys_catalog.cc:458] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:21:20.862110 20665 mvcc.cc:205] Tried to move back new op lower bound from 7316698238404194304 to 7316698238212190208. Current Snapshot: MvccSnapshot[applied={T|T < 7316698238404194304}]
I20260809 19:21:20.862192 20641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:20.862267 20641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:20.862416 20643 sys_catalog.cc:455] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "fbb92827f9434fa18ac3d334b7abcd44" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 34371 } } peers { permanent_uuid: "508f35eb0fbb425b9c0f975d8ce0bcb4" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38385 } attrs { promote: false } } peers { permanent_uuid: "da147739e9334c93922217ef735cd93a" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38053 } attrs { promote: false } } }
I20260809 19:21:20.862502 20643 sys_catalog.cc:458] T 00000000000000000000000000000000 P da147739e9334c93922217ef735cd93a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:21:20.866676 20648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.863840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44323 (local address 127.0.0.1:52394)
0809 19:21:20.863882 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.863892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:20.863930 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:20.864159 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:20.864160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:20.864162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:20.864202 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.864203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.864931 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.864932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.865706 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.865708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.866470 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.866471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.866530 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.866535 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:20.866577 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 19:21:20.866613 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:21:20.866971 20646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.863895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:41079 (local address 127.0.0.1:43934)
0809 19:21:20.863925 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.863934 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:20.863978 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:20.864426 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:20.864427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:20.864430 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:20.864462 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.864464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.865225 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.865225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.866199 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.866201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.866739 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.866740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.866799 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.866802 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:20.866848 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:21:20.866890 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 19:21:20.866971 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.863905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.9.91.66:44323)
0809 19:21:20.864011 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:20.864012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:20.864020 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:20.864046 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:20.864047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:20.864057 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:20.864265 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:20.864305 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.864861 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.865785 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.866357 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.866425 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.866438 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:20.866833 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 19:21:20.866882 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260809 19:21:20.867161 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.864183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.9.91.67:41079)
0809 19:21:20.864291 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:20.864293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:20.864301 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:20.864326 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:20.864327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:20.864336 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:20.864418 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:20.864560 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.865152 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.866274 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.866665 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.866728 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.866740 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:20.866975 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 19:21:20.867033 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
I20260809 19:21:20.867300 20225 tablet_service.cc:1467] Processing CreateTablet for tablet 6c822703eb124af4b051bba8a9052b14 (DEFAULT_TABLE table=test-table [id=026b64f8eef24736b195a119d11027a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:20.867367 20441 tablet_service.cc:1467] Processing CreateTablet for tablet 6c822703eb124af4b051bba8a9052b14 (DEFAULT_TABLE table=test-table [id=026b64f8eef24736b195a119d11027a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:20.867439 20225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c822703eb124af4b051bba8a9052b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:20.867471 20441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c822703eb124af4b051bba8a9052b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:20.868132 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.863982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42883 (local address 127.0.0.1:37590)
0809 19:21:20.864107 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.864117 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:20.864156 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:20.865065 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:20.865066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:20.865068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:20.865105 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.865107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.865940 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.865941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.867005 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.867008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.867908 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.867909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.867973 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.867977 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:20.868030 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:21:20.868069 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":91,"threads_started":1}
I20260809 19:21:20.868147 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.864042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.9.91.65:42883)
0809 19:21:20.864959 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:20.864961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:20.864966 (+ 5us) server_negotiation.cc:407] Connection header received
0809 19:21:20.864987 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:20.864988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:20.864997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:20.865048 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:20.865154 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.865863 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.867380 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.867828 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.867894 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.867907 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:20.868036 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 19:21:20.868083 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":897,"thread_start_us":56,"threads_started":1}
I20260809 19:21:20.868873 20678 tablet_bootstrap.cc:492] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Bootstrap starting.
I20260809 19:21:20.869046 20007 tablet_service.cc:1467] Processing CreateTablet for tablet 6c822703eb124af4b051bba8a9052b14 (DEFAULT_TABLE table=test-table [id=026b64f8eef24736b195a119d11027a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:20.869164 20007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c822703eb124af4b051bba8a9052b14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:20.869343 20678 tablet_bootstrap.cc:654] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:20.869757 20679 tablet_bootstrap.cc:492] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: Bootstrap starting.
I20260809 19:21:20.870074 20678 tablet_bootstrap.cc:492] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: No bootstrap required, opened a new log
I20260809 19:21:20.870131 20679 tablet_bootstrap.cc:654] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:20.870208 20678 ts_tablet_manager.cc:1397] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:20.870383 20678 raft_consensus.cc:348] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.870442 20682 tablet_bootstrap.cc:492] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba: Bootstrap starting.
I20260809 19:21:20.870558 20678 raft_consensus.cc:374] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:20.870597 20678 raft_consensus.cc:729] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bba31429c34198b9b265c107d9314d, State: Initialized, Role: FOLLOWER
I20260809 19:21:20.870639 20678 consensus_queue.cc:260] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.870811 20678 ts_tablet_manager.cc:1428] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:20.870824 20682 tablet_bootstrap.cc:654] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:20.871191 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.870967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.67:34083)
0809 19:21:20.871112 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.871158 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260809 19:21:20.871341 20682 tablet_bootstrap.cc:492] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba: No bootstrap required, opened a new log
I20260809 19:21:20.871335 20546 heartbeater.cc:503] Master 127.9.91.125:38385 was elected leader, sending a full tablet report...
I20260809 19:21:20.871387 20682 ts_tablet_manager.cc:1397] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:20.871490 20682 raft_consensus.cc:348] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.871551 20682 raft_consensus.cc:374] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:20.871547 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.871377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.67:50747)
0809 19:21:20.871500 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.871524 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260809 19:21:20.871569 20682 raft_consensus.cc:729] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ee6bfb6754406e9e690f6428fa6fba, State: Initialized, Role: FOLLOWER
I20260809 19:21:20.871614 20682 consensus_queue.cc:260] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.871754 20682 ts_tablet_manager.cc:1428] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:20.872184 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.871961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.65:40443)
0809 19:21:20.872104 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.872149 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":97,"threads_started":1}
I20260809 19:21:20.872246 20110 heartbeater.cc:503] Master 127.9.91.125:38385 was elected leader, sending a full tablet report...
I20260809 19:21:20.872614 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.872408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.65:53147)
0809 19:21:20.872561 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.872587 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":114,"threads_started":1}
I20260809 19:21:20.872783 20679 tablet_bootstrap.cc:492] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: No bootstrap required, opened a new log
I20260809 19:21:20.872838 20679 ts_tablet_manager.cc:1397] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:21:20.872972 20679 raft_consensus.cc:348] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.873028 20679 raft_consensus.cc:374] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:20.873046 20679 raft_consensus.cc:729] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0195befba7b040dfaed2d0055fc2405e, State: Initialized, Role: FOLLOWER
I20260809 19:21:20.873085 20679 consensus_queue.cc:260] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.873232 20679 ts_tablet_manager.cc:1428] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:20.873579 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.873369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.66:49625)
0809 19:21:20.873500 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.873535 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260809 19:21:20.873665 20337 heartbeater.cc:503] Master 127.9.91.125:38385 was elected leader, sending a full tablet report...
I20260809 19:21:20.874001 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.873759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.66:57041)
0809 19:21:20.873900 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.873969 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":77,"threads_started":1}
W20260809 19:21:20.874199 20547 tablet.cc:2369] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:21:20.983479 20698 raft_consensus.cc:482] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:20.983572 20698 raft_consensus.cc:504] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.983868 20698 leader_election.cc:288] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079), e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65:42883)
I20260809 19:21:20.986672 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.984126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:36467 (local address 127.9.91.67:41079)
0809 19:21:20.984176 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:20.984178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:20.984191 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:21:20.984235 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:20.984236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:20.984248 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:20.984295 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:20.984462 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.984955 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.985957 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.986318 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.986379 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.986391 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:20.986571 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 19:21:20.986610 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:21:20.986660 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.984233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:60987 (local address 127.9.91.65:42883)
0809 19:21:20.984283 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:20.984285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:20.984294 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:20.984318 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:20.984319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:20.984330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:20.984383 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:20.984521 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.985068 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.985962 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.986319 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.986379 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.986391 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:20.986545 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:21:20.986588 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:21:20.986672 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.984135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42883 (local address 127.9.91.66:60987)
0809 19:21:20.984174 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.984183 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:20.984229 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:20.984398 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:20.984399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:20.984400 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:20.984458 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.984460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.985157 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.985158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.985873 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.985875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.986397 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.986398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.986476 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.986479 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:20.986527 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:21:20.986564 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:21:20.986675 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.984038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:41079 (local address 127.9.91.66:36467)
0809 19:21:20.984097 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.984109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:20.984170 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:20.984305 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:20.984306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:20.984309 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:20.984393 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.984395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.985064 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.985065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.985862 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:20.985865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:20.986435 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:20.986436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:20.986496 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:20.986500 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:20.986554 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:21:20.986592 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:21:20.987002 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c822703eb124af4b051bba8a9052b14" candidate_uuid: "0195befba7b040dfaed2d0055fc2405e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bba31429c34198b9b265c107d9314d" is_pre_election: true
I20260809 19:21:20.987013 20040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c822703eb124af4b051bba8a9052b14" candidate_uuid: "0195befba7b040dfaed2d0055fc2405e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" is_pre_election: true
I20260809 19:21:20.987097 20472 raft_consensus.cc:2454] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0195befba7b040dfaed2d0055fc2405e in term 0.
I20260809 19:21:20.987157 20040 raft_consensus.cc:2454] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0195befba7b040dfaed2d0055fc2405e in term 0.
I20260809 19:21:20.987265 20143 leader_election.cc:302] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0195befba7b040dfaed2d0055fc2405e, d1bba31429c34198b9b265c107d9314d; no voters:
I20260809 19:21:20.987411 20698 raft_consensus.cc:2781] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:20.987452 20698 raft_consensus.cc:482] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:20.987473 20698 raft_consensus.cc:3037] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:20.988155 20698 raft_consensus.cc:504] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.988265 20698 leader_election.cc:288] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [CANDIDATE]: Term 1 election: Requested vote from peers d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079), e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65:42883)
I20260809 19:21:20.988395 20472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c822703eb124af4b051bba8a9052b14" candidate_uuid: "0195befba7b040dfaed2d0055fc2405e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bba31429c34198b9b265c107d9314d"
I20260809 19:21:20.988435 20040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c822703eb124af4b051bba8a9052b14" candidate_uuid: "0195befba7b040dfaed2d0055fc2405e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ee6bfb6754406e9e690f6428fa6fba"
I20260809 19:21:20.988472 20472 raft_consensus.cc:3037] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:20.988500 20040 raft_consensus.cc:3037] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:20.989039 20472 raft_consensus.cc:2454] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0195befba7b040dfaed2d0055fc2405e in term 1.
I20260809 19:21:20.989168 20040 raft_consensus.cc:2454] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0195befba7b040dfaed2d0055fc2405e in term 1.
I20260809 19:21:20.989197 20143 leader_election.cc:302] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0195befba7b040dfaed2d0055fc2405e, d1bba31429c34198b9b265c107d9314d; no voters:
I20260809 19:21:20.989276 20698 raft_consensus.cc:2781] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:20.989449 20698 raft_consensus.cc:686] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 1 LEADER]: Becoming Leader. State: Replica: 0195befba7b040dfaed2d0055fc2405e, State: Running, Role: LEADER
I20260809 19:21:20.989521 20698 consensus_queue.cc:237] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } }
I20260809 19:21:20.989811 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:20.989695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.66:35889)
0809 19:21:20.989750 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:20.989786 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:21:20.990265 19540 catalog_manager.cc:5495] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0195befba7b040dfaed2d0055fc2405e (127.9.91.66). New cstate: current_term: 1 leader_uuid: "0195befba7b040dfaed2d0055fc2405e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0195befba7b040dfaed2d0055fc2405e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:21.010334 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:21.008203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52398 (local address 127.9.91.66:44323)
0809 19:21:21.008261 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:21.008263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:21.008271 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:21.008338 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:21.008339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:21.008355 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:21.008410 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:21.008520 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:21.008974 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:21.009677 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:21.009859 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:21.009912 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:21.010046 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:21.010144 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:21.010206 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:21.010249 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 19:21:21.010269 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:21:21.011296 20472 raft_consensus.cc:1261] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Refusing update from remote peer 0195befba7b040dfaed2d0055fc2405e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:21.011338 20040 raft_consensus.cc:1261] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 FOLLOWER]: Refusing update from remote peer 0195befba7b040dfaed2d0055fc2405e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:21.011571 20698 consensus_queue.cc:1035] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bba31429c34198b9b265c107d9314d" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:21.011715 20702 consensus_queue.cc:1035] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ee6bfb6754406e9e690f6428fa6fba" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 42883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:21.012421 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:21.012257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.67:44543)
0809 19:21:21.012361 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:21.012391 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 19:21:21.012487 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:21.012368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.65:40225)
0809 19:21:21.012418 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:21.012452 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:21:21.013819 20699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:21.013734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.66:56755)
0809 19:21:21.013769 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:21.013793 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 19:21:21.013890 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:21.013803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.67:49005)
0809 19:21:21.013840 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:21.013864 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 19:21:21.014616 20695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:21.014479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:34371 (local address 127.9.91.65:33687)
0809 19:21:21.014563 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:21.014588 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:34371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 19:21:21.015829 19539 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:21.015928 19539 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:21.017278 19539 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 508f35eb0fbb425b9c0f975d8ce0bcb4: Sending DeleteTablet for 3 replicas of tablet 6c822703eb124af4b051bba8a9052b14
I20260809 19:21:21.017609 20007 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c822703eb124af4b051bba8a9052b14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:21 UTC) from {username='oryx', principal='oryx/127.9.91.125@KRBTEST.COM'} at 127.0.0.1:37590
I20260809 19:21:21.017609 20441 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c822703eb124af4b051bba8a9052b14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:21 UTC) from {username='oryx', principal='oryx/127.9.91.125@KRBTEST.COM'} at 127.0.0.1:43934
I20260809 19:21:21.017693 20225 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c822703eb124af4b051bba8a9052b14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:21 UTC) from {username='oryx', principal='oryx/127.9.91.125@KRBTEST.COM'} at 127.0.0.1:52394
I20260809 19:21:21.017884 20725 tablet_replica.cc:330] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: stopping tablet replica
I20260809 19:21:21.017938 20722 tablet_replica.cc:330] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba: stopping tablet replica
I20260809 19:21:21.017962 20725 raft_consensus.cc:2229] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:21.018003 20722 raft_consensus.cc:2229] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:21.018044 20722 raft_consensus.cc:2258] T 6c822703eb124af4b051bba8a9052b14 P e6ee6bfb6754406e9e690f6428fa6fba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:21.018102 20725 raft_consensus.cc:2258] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:21.018158 20721 tablet_replica.cc:330] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: stopping tablet replica
I20260809 19:21:21.018222 20721 raft_consensus.cc:2229] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:21.018309 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 19770
I20260809 19:21:21.018450 20725 ts_tablet_manager.cc:1905] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:21.019048 20721 raft_consensus.cc:2258] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:21.020289 20725 ts_tablet_manager.cc:1918] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:21.020416 20725 log.cc:1196] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/6c822703eb124af4b051bba8a9052b14
I20260809 19:21:21.020723 20725 ts_tablet_manager.cc:1939] T 6c822703eb124af4b051bba8a9052b14 P 0195befba7b040dfaed2d0055fc2405e: Deleting consensus metadata
I20260809 19:21:21.021518 19513 catalog_manager.cc:4841] TS 0195befba7b040dfaed2d0055fc2405e (127.9.91.66:44323): tablet 6c822703eb124af4b051bba8a9052b14 (table test-table [id=026b64f8eef24736b195a119d11027a7]) successfully deleted
I20260809 19:21:21.021716 20721 ts_tablet_manager.cc:1905] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:21.023247 20721 ts_tablet_manager.cc:1918] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:21.023311 20721 log.cc:1196] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/6c822703eb124af4b051bba8a9052b14
I20260809 19:21:21.023547 20721 ts_tablet_manager.cc:1939] T 6c822703eb124af4b051bba8a9052b14 P d1bba31429c34198b9b265c107d9314d: Deleting consensus metadata
I20260809 19:21:21.023864 19511 catalog_manager.cc:4841] TS d1bba31429c34198b9b265c107d9314d (127.9.91.67:41079): tablet 6c822703eb124af4b051bba8a9052b14 (table test-table [id=026b64f8eef24736b195a119d11027a7]) successfully deleted
I20260809 19:21:21.032159 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:21.031993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:42883 (local address 127.0.0.1:37604)
0809 19:21:21.032098 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:21.032125 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:42883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 19:21:21.032308 19511 catalog_manager.cc:4584] TS e6ee6bfb6754406e9e690f6428fa6fba (127.9.91.65:42883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c822703eb124af4b051bba8a9052b14: Network error: Client connection negotiation failed: client connection to 127.9.91.65:42883: connect: Connection refused (error 111)
I20260809 19:21:21.032330 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 20117
I20260809 19:21:21.054282 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 20345
I20260809 19:21:21.067059 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 19329
I20260809 19:21:21.077553 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 19623
2026-08-09T19:21:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4781 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:21 dist-test-slave-1vmg krb5kdc[20728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:21 dist-test-slave-1vmg krb5kdc[20728](info): set up 2 sockets
Aug 09 19:21:21 dist-test-slave-1vmg krb5kdc[20728](info): commencing operation
krb5kdc: starting...
W20260809 19:21:22.210532 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:22Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab.
Entry for principal oryx/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab.
I20260809 19:21:22.323547 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:39427
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab
--principal=oryx/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:39427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:22.402359 20742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:22.402544 20742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:22.402563 20742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:22.403646 20742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:22.403678 20742 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 19:21:22.403689 20742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:22.403702 20742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:22.403716 20742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:22.403726 20742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:22.405162 20742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:39427
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.126.keytab
--principal=oryx/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:39427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:22.405340 20742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:22.405501 20742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:21:22.410030 20742 server_base.cc:1034] running on GCE node
W20260809 19:21:22.409996 20749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:22.410028 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:22.409991 20757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.410418 20742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:22.410686 20742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:22.411818 20742 hybrid_clock.cc:648] HybridClock initialized: now 1786303282411806 us; error 30 us; skew 500 ppm
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:22.414068 20742 init.cc:377] Logged in from keytab as oryx/127.9.91.126@KRBTEST.COM (short username oryx)
I20260809 19:21:22.414445 20742 webserver.cc:460] Webserver started at http://127.9.91.126:42537/ using document root <none> and password file <none>
I20260809 19:21:22.414628 20742 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:22.414677 20742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:22.414796 20742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:22.415813 20742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "57925294b7194aaaa678d71180676650"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "5ca1a4d38cb6273a83aa4fe83e73b05b"
server_key_iv: "e08b59945c5ce95cf346909958e81e56"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.416127 20742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "57925294b7194aaaa678d71180676650"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "5ca1a4d38cb6273a83aa4fe83e73b05b"
server_key_iv: "e08b59945c5ce95cf346909958e81e56"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.417424 20742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:22.418073 20765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.418216 20742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:22.418295 20742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "57925294b7194aaaa678d71180676650"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "5ca1a4d38cb6273a83aa4fe83e73b05b"
server_key_iv: "e08b59945c5ce95cf346909958e81e56"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.418357 20742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:22.432039 20742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:22.433529 20742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:22.433656 20742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:22.437997 20742 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:39427
I20260809 19:21:22.438054 20866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:39427 every 8 connection(s)
I20260809 19:21:22.438331 20742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:22.438900 20867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:22.440999 20867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Bootstrap starting.
I20260809 19:21:22.441567 20867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:22.441848 20867 log.cc:824] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:22.442451 20867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: No bootstrap required, opened a new log
I20260809 19:21:22.444417 20867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57925294b7194aaaa678d71180676650" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39427 } }
I20260809 19:21:22.444542 20867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:22.444564 20867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57925294b7194aaaa678d71180676650, State: Initialized, Role: FOLLOWER
I20260809 19:21:22.444700 20867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57925294b7194aaaa678d71180676650" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39427 } }
I20260809 19:21:22.444763 20867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:22.444787 20867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:22.444831 20867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:22.445314 20867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57925294b7194aaaa678d71180676650" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39427 } }
I20260809 19:21:22.445405 20867 leader_election.cc:302] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57925294b7194aaaa678d71180676650; no voters:
I20260809 19:21:22.445559 20867 leader_election.cc:288] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:22.445596 20870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:22.445719 20870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [term 1 LEADER]: Becoming Leader. State: Replica: 57925294b7194aaaa678d71180676650, State: Running, Role: LEADER
I20260809 19:21:22.445801 20867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:22.445802 20870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57925294b7194aaaa678d71180676650" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39427 } }
I20260809 19:21:22.446168 20871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57925294b7194aaaa678d71180676650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57925294b7194aaaa678d71180676650" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39427 } } }
I20260809 19:21:22.446180 20872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57925294b7194aaaa678d71180676650. Latest consensus state: current_term: 1 leader_uuid: "57925294b7194aaaa678d71180676650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57925294b7194aaaa678d71180676650" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 39427 } } }
I20260809 19:21:22.446262 20871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:22.446272 20872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:22.446440 20880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:22.446957 20880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:22.447551 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 20742
I20260809 19:21:22.447680 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:22.447866 9581 external_mini_cluster.cc:1428] Setting key 768b8ef9a69c0d10a98065c214599a71
I20260809 19:21:22.448719 20880 catalog_manager.cc:1347] Generated new cluster ID: c12fc6e7e30d4cd88ce966328fc0b2d1
I20260809 19:21:22.448771 20880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:22.454290 20880 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:22.454767 20880 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:22.458014 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.448296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33808 (local address 127.9.91.126:39427)
0809 19:21:22.448468 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.448477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.448508 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:21:22.448617 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.448623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.448680 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.448806 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.449463 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.450369 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.451152 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.451582 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.451673 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.453784 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.453788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.453789 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.453800 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.456523 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.456893 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.456896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.456896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.457001 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.457174 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.457175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.457175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.457431 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.457596 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.457659 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.457803 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260809 19:21:22.459080 20880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Generated new TSK 0
I20260809 19:21:22.459233 20880 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab.
Entry for principal oryx/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab.
I20260809 19:21:22.493355 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39427
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab
--principal=oryx/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:22.595046 20904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:22.595255 20904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:22.595281 20904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:22.596385 20904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:22.596423 20904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:22.596442 20904 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:21:22.596458 20904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:22.597939 20904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab
--principal=oryx/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:39427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:22.598148 20904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:22.598358 20904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:22.603125 20914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:22.603132 20915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.603264 20904 server_base.cc:1034] running on GCE node
W20260809 19:21:22.603159 20920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.603518 20904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:22.603739 20904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:22.604895 20904 hybrid_clock.cc:648] HybridClock initialized: now 1786303282604874 us; error 36 us; skew 500 ppm
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:22.607317 20904 init.cc:377] Logged in from keytab as oryx/127.9.91.65@KRBTEST.COM (short username oryx)
I20260809 19:21:22.607705 20904 webserver.cc:460] Webserver started at http://127.9.91.65:39831/ using document root <none> and password file <none>
I20260809 19:21:22.607910 20904 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:22.607954 20904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:22.608069 20904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:22.609017 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "3411571940704655985281caf5335b63"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "dc8d0e194bc56ea1ec7fb4f2361cd700"
server_key_iv: "06a10ce34416bbd8a2f549ca95ad9233"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.609329 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "3411571940704655985281caf5335b63"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "dc8d0e194bc56ea1ec7fb4f2361cd700"
server_key_iv: "06a10ce34416bbd8a2f549ca95ad9233"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.610487 20904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:22.611132 20931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.611253 20904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:22.611330 20904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "3411571940704655985281caf5335b63"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "dc8d0e194bc56ea1ec7fb4f2361cd700"
server_key_iv: "06a10ce34416bbd8a2f549ca95ad9233"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.611389 20904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:22.619161 20904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:22.620700 20904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:22.620817 20904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:22.621243 20904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:22.621295 20904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.621317 20904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:22.621342 20904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:22.627925 20904 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:35717
I20260809 19:21:22.627988 21111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:35717 every 8 connection(s)
I20260809 19:21:22.628259 20904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:22.630822 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.621647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:55431 (local address 127.9.91.126:39427)
0809 19:21:22.621708 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.621711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.621883 (+ 172us) server_negotiation.cc:407] Connection header received
0809 19:21:22.622353 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.622354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.622368 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.622450 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.623328 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.624082 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.625507 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.625823 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.625893 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.628209 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.628210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.628211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.628218 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.629594 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.629949 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.629950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.629951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.630011 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.630224 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.630225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.630226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.630339 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.630394 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.630646 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.630734 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 19:21:22.630980 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.621618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39427 (local address 127.9.91.65:55431)
0809 19:21:22.621753 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.621789 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.622257 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.622559 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.622564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.622908 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:22.623214 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.623220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.624166 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.624168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.625420 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.625423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.625923 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.625924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.626066 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.626501 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:22.626511 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:22.628123 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:22.629668 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.629675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.629677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.629882 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.630065 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.630066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.630066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.630169 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.630392 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:22.630399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:22.630528 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 19:21:22.630612 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 19:21:22.630758 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":71,"threads_started":1}
I20260809 19:21:22.631456 21112 heartbeater.cc:347] Connected to a master server at 127.9.91.126:39427
I20260809 19:21:22.631543 21112 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:22.631718 21112 heartbeater.cc:511] Master 127.9.91.126:39427 requested a full tablet report, sending...
I20260809 19:21:22.632233 20792 ts_manager.cc:194] Registered new tserver with Master: 3411571940704655985281caf5335b63 (127.9.91.65:35717)
I20260809 19:21:22.632891 20792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:55431
I20260809 19:21:22.637332 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.632496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:45327 (local address 127.9.91.126:39427)
0809 19:21:22.632529 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.632530 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.632646 (+ 116us) server_negotiation.cc:407] Connection header received
0809 19:21:22.632699 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.632700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.632709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.632753 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.633067 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.633512 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.634166 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.634359 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.634409 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.635246 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.635247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.635247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.635254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.636396 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.636680 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.636681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.636681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.636730 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.636885 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.636886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.636886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.636981 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.637034 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.637182 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.637245 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:21:22.637346 21120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.632440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39427 (local address 127.9.91.65:45327)
0809 19:21:22.632587 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.632595 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.632647 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.632756 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.632756 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.632916 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:22.632969 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.632970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.633601 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.633602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.634100 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.634101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.634415 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.634415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.634477 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.634833 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:22.634837 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:22.635172 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:22.636468 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.636469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.636469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.636622 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.636780 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.636780 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.636781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.636836 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.637040 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:22.637041 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:22.637084 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:21:22.637153 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:21:22.637247 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260809 19:21:22.638113 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 20904
I20260809 19:21:22.638231 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:22.638438 9581 external_mini_cluster.cc:1428] Setting key f6a7243361ef448bc6559ed81c36fd2a
WARNING: no policy specified for oryx/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab.
Entry for principal oryx/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab.
I20260809 19:21:22.661161 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39427
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab
--principal=oryx/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:22.740911 21125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:22.741101 21125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:22.741125 21125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:22.742242 21125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:22.742285 21125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:22.742311 21125 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:21:22.742345 21125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:22.743856 21125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab
--principal=oryx/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:39427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:22.744068 21125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:22.744266 21125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:22.749159 21132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:22.749178 21138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.749225 21125 server_base.cc:1034] running on GCE node
W20260809 19:21:22.749548 21130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.749760 21125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:22.750003 21125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:22.751150 21125 hybrid_clock.cc:648] HybridClock initialized: now 1786303282751126 us; error 35 us; skew 500 ppm
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:22.753337 21125 init.cc:377] Logged in from keytab as oryx/127.9.91.66@KRBTEST.COM (short username oryx)
I20260809 19:21:22.753696 21125 webserver.cc:460] Webserver started at http://127.9.91.66:46845/ using document root <none> and password file <none>
I20260809 19:21:22.753875 21125 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:22.753921 21125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:22.754042 21125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:22.755028 21125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "66b94ca1d24542a0999fd4f9a18eafda"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "7417239df2b63f78e0f96e78dd38f725"
server_key_iv: "ebaf23f20a9ed0bf4dac8f776f7a4a90"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.755339 21125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "66b94ca1d24542a0999fd4f9a18eafda"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "7417239df2b63f78e0f96e78dd38f725"
server_key_iv: "ebaf23f20a9ed0bf4dac8f776f7a4a90"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.756631 21125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:22.757287 21146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.757448 21125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:22.757525 21125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "66b94ca1d24542a0999fd4f9a18eafda"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "7417239df2b63f78e0f96e78dd38f725"
server_key_iv: "ebaf23f20a9ed0bf4dac8f776f7a4a90"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.757582 21125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:22.789763 21125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:22.791334 21125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:22.791457 21125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:22.791886 21125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:22.791926 21125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.791955 21125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:22.791971 21125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.797978 21125 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:41317
I20260809 19:21:22.798094 21326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:41317 every 8 connection(s)
I20260809 19:21:22.798324 21125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:22.801035 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.792315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:44387 (local address 127.9.91.126:39427)
0809 19:21:22.792381 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.792383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.792575 (+ 192us) server_negotiation.cc:407] Connection header received
0809 19:21:22.793021 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.793023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.793037 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.793089 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.793860 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.794596 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.795747 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.796042 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.796117 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.798878 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.798879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.798879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.798885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.800053 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.800387 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.800388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.800388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.800435 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.800589 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.800589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.800589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.800688 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.800732 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.800915 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.800975 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:21:22.801167 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.792268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39427 (local address 127.9.91.66:44387)
0809 19:21:22.792456 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.792494 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.792925 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.793151 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.793155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.793486 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:22.793750 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.793756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.794704 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.794706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.795678 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.795681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.796135 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.796136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.796271 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.797210 (+ 939us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:22.797219 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:22.798801 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:22.800127 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.800133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.800135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.800335 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.800471 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.800471 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.800472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.800550 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.800728 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:22.800731 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:22.800830 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 19:21:22.800901 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:21:22.801006 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":70,"threads_started":1}
I20260809 19:21:22.801586 21327 heartbeater.cc:347] Connected to a master server at 127.9.91.126:39427
I20260809 19:21:22.801662 21327 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:22.801832 21327 heartbeater.cc:511] Master 127.9.91.126:39427 requested a full tablet report, sending...
I20260809 19:21:22.802199 20794 ts_manager.cc:194] Registered new tserver with Master: 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317)
I20260809 19:21:22.802585 20794 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:44387
I20260809 19:21:22.805605 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 21125
I20260809 19:21:22.805722 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:22.805923 9581 external_mini_cluster.cc:1428] Setting key 5e3d09b7d89c1552cad34452f712dd0f
I20260809 19:21:22.807585 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.802857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:33509 (local address 127.9.91.126:39427)
0809 19:21:22.802893 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.802895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.803023 (+ 128us) server_negotiation.cc:407] Connection header received
0809 19:21:22.803050 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.803051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.803060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.803123 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.803396 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.803843 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.804446 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.804604 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.804642 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.805298 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.805299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.805299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.805306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.806539 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.806836 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.806837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.806837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.806893 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.807101 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.807102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.807102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.807242 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.807299 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.807435 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.807499 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:21:22.807590 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.802820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39427 (local address 127.9.91.66:33509)
0809 19:21:22.802959 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.802968 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.803020 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.803125 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.803125 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.803272 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:22.803320 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.803321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.803914 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.803915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.804393 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.804394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.804654 (+ 260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.804654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.804696 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.804995 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:22.804998 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:22.805233 (+ 235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:22.806609 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.806610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.806610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.806767 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.806989 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.806989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.806990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.807049 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.807302 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:22.807303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:22.807347 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 19:21:22.807406 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:21:22.807500 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab.
Entry for principal oryx/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab.
I20260809 19:21:22.828787 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:39427
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab
--principal=oryx/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:22.908947 21337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:22.909137 21337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:22.909160 21337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:22.910238 21337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:22.910277 21337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:22.910298 21337 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 19:21:22.910322 21337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:22.911765 21337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab
--principal=oryx/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.91.126:39427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:22.911937 21337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:22.912119 21337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:22.916852 21344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:22.916862 21342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.916994 21337 server_base.cc:1034] running on GCE node
W20260809 19:21:22.916836 21347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:22.917266 21337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:22.917497 21337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:22.918630 21337 hybrid_clock.cc:648] HybridClock initialized: now 1786303282918609 us; error 43 us; skew 500 ppm
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:22.921036 21337 init.cc:377] Logged in from keytab as oryx/127.9.91.67@KRBTEST.COM (short username oryx)
I20260809 19:21:22.921466 21337 webserver.cc:460] Webserver started at http://127.9.91.67:36777/ using document root <none> and password file <none>
I20260809 19:21:22.921663 21337 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:22.921710 21337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:22.921835 21337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:22.922806 21337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "bade9033e47b420a92b19bc3d78ecf2c"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "80e305f480215f971f45ff88e68b7dec"
server_key_iv: "e9b2ba6fc553f227292c100c6ef71108"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.923174 21337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "bade9033e47b420a92b19bc3d78ecf2c"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "80e305f480215f971f45ff88e68b7dec"
server_key_iv: "e9b2ba6fc553f227292c100c6ef71108"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.924455 21337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:22.925089 21356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.925239 21337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.925329 21337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "bade9033e47b420a92b19bc3d78ecf2c"
format_stamp: "Formatted at 2026-08-09 19:21:22 on dist-test-slave-1vmg"
server_key: "80e305f480215f971f45ff88e68b7dec"
server_key_iv: "e9b2ba6fc553f227292c100c6ef71108"
server_key_version: "encryptionkey@0"
I20260809 19:21:22.925392 21337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:22.941903 21337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:22.943554 21337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:22.943677 21337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:22.944094 21337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:22.944134 21337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:22.944172 21337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:22.944197 21337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:22.950984 21337 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:43541
I20260809 19:21:22.951054 21552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:43541 every 8 connection(s)
I20260809 19:21:22.951352 21337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:22.952966 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 21337
I20260809 19:21:22.953076 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:22.953286 9581 external_mini_cluster.cc:1428] Setting key aac92fdeaa0b75bd356fd5a2cca157c6
I20260809 19:21:22.953811 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.944493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:53427 (local address 127.9.91.126:39427)
0809 19:21:22.944537 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.944538 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.944718 (+ 180us) server_negotiation.cc:407] Connection header received
0809 19:21:22.945115 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.945116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.945154 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.945209 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.945974 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.946808 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.947912 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.948199 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.948273 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.951122 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.951124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.951124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.951132 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.952511 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.952875 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.952876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.952876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.952922 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.953165 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.953166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.953166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.953271 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.953320 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.953631 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.953721 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:21:22.953962 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.944489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39427 (local address 127.9.91.67:53427)
0809 19:21:22.944616 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.944648 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.945027 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.945275 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.945279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.945547 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:22.945865 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.945871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.946903 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.946904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.947838 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.947840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.948283 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.948284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.948414 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.948843 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:22.948853 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:22.951040 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:22.952584 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.952589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.952591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.952810 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.952973 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.952973 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.952974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.953106 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.953330 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:22.953337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:22.953488 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 19:21:22.953596 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 19:21:22.953762 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":65,"threads_started":1}
I20260809 19:21:22.954391 21553 heartbeater.cc:347] Connected to a master server at 127.9.91.126:39427
I20260809 19:21:22.954471 21553 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:22.954624 21553 heartbeater.cc:511] Master 127.9.91.126:39427 requested a full tablet report, sending...
I20260809 19:21:22.955016 20792 ts_manager.cc:194] Registered new tserver with Master: bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541)
I20260809 19:21:22.955492 20792 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:53427
I20260809 19:21:22.960043 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.955255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39427 (local address 127.9.91.67:38181)
0809 19:21:22.955365 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.955375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.955430 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.955532 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.955533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.955733 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:22.955788 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.955790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.956540 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.956540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.957024 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.957025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.957298 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.957299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.957339 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.957691 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:22.957696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:22.957967 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:22.959189 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.959191 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.959191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.959362 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.959549 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:22.959549 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:22.959550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:22.959607 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:22.959787 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:22.959787 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:22.959820 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 19:21:22.959865 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:22.959940 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":62,"threads_started":1}
I20260809 19:21:22.960048 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.955315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:38181 (local address 127.9.91.126:39427)
0809 19:21:22.955352 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.955353 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.955417 (+ 64us) server_negotiation.cc:407] Connection header received
0809 19:21:22.955478 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.955479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.955486 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.955526 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.955863 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.956465 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.957078 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.957252 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.957293 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.958040 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.958041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.958041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.958045 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.959055 (+ 1010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.959442 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.959443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.959443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.959495 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.959653 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.959653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.959653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.959735 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.959788 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.959886 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.959968 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:21:22.964872 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:21:22.985440 20901 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:22.980596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.9.91.126:39427)
0809 19:21:22.980646 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.980647 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.980700 (+ 53us) server_negotiation.cc:407] Connection header received
0809 19:21:22.980754 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.980754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.980762 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.980809 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.981305 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.981755 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.982426 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.982599 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.982642 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.985374 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33814: BlockingRecv error: recv got EOF from 127.0.0.1:33814 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 19:21:22 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:22.991633 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.986120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.9.91.126:39427)
0809 19:21:22.986160 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.986162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.986171 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:22.986215 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.986216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.986223 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.986264 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:22.986673 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.987155 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.987754 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.987922 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.987961 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.989709 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:22.989710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:22.989710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:22.989717 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:22.990796 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.991133 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.991134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.991134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.991194 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:22.991318 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:22.991318 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:22.991319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:22.991401 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:22.991463 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.991528 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.991579 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:21:22.992908 20792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:22.994398 20792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:22.999962 21586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.996477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35717 (local address 127.0.0.1:50534)
0809 19:21:22.996753 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.996764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.996829 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.997039 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.997042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.997044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:22.997099 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.997100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.998091 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.998092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.998876 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.998878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.999668 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.999668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.999768 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.999781 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:21:22.999846 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 19:21:22.999895 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":70,"threads_started":1}
I20260809 19:21:23.000294 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.996598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.9.91.65:35717)
0809 19:21:22.996888 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.996892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.996900 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:22.996936 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.996938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.996951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.997028 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:22.997189 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.998018 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.998979 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.999597 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.999680 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:22.999768 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:22.999864 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:21:22.999904 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":90,"threads_started":1}
I20260809 19:21:23.000720 21587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.996449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:43541 (local address 127.0.0.1:46016)
0809 19:21:22.996718 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.996760 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.996828 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.997160 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.997161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.997163 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:22.997203 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.997204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.998404 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.998405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.999546 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.999548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.000508 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.000509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.000570 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.000574 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.000621 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.000656 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":47,"threads_started":1}
I20260809 19:21:23.000784 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.996548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46016 (local address 127.9.91.67:43541)
0809 19:21:22.996790 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.996795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.996806 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:22.996934 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.996936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.996951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.997028 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:22.997287 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.998306 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.999628 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.000334 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.000401 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.000474 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.000649 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.000696 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":75,"threads_started":1}
I20260809 19:21:23.000957 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.996632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41317 (local address 127.0.0.1:44888)
0809 19:21:22.997115 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:22.997124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:22.997188 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:22.997481 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:22.997482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:22.997484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:22.997521 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.997523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.998683 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.998684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:22.999787 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:22.999789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.000737 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.000738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.000796 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.000801 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.000846 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.000883 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":10,"thread_start_us":74,"threads_started":1}
I20260809 19:21:23.000979 21001 tablet_service.cc:1467] Processing CreateTablet for tablet 6835da24886f4b018b1ae2a33cde6b61 (DEFAULT_TABLE table=test-table [id=0e19619d6e6e4442b9dd46301ad64f43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.001240 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:22.996816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.9.91.66:41317)
0809 19:21:22.996979 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:22.996983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:22.997239 (+ 256us) server_negotiation.cc:407] Connection header received
0809 19:21:22.997289 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:22.997291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:22.997306 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:22.997361 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:22.997584 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:22.998604 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:22.999875 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.000565 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.000634 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.000702 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.001097 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.001137 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260809 19:21:23.001375 21001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6835da24886f4b018b1ae2a33cde6b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.001287 21438 tablet_service.cc:1467] Processing CreateTablet for tablet 6835da24886f4b018b1ae2a33cde6b61 (DEFAULT_TABLE table=test-table [id=0e19619d6e6e4442b9dd46301ad64f43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.001612 21438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6835da24886f4b018b1ae2a33cde6b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.001839 21218 tablet_service.cc:1467] Processing CreateTablet for tablet 6835da24886f4b018b1ae2a33cde6b61 (DEFAULT_TABLE table=test-table [id=0e19619d6e6e4442b9dd46301ad64f43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.002134 21218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6835da24886f4b018b1ae2a33cde6b61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.004746 21596 tablet_bootstrap.cc:492] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Bootstrap starting.
I20260809 19:21:23.005291 21598 tablet_bootstrap.cc:492] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Bootstrap starting.
I20260809 19:21:23.005458 21596 tablet_bootstrap.cc:654] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.005710 21599 tablet_bootstrap.cc:492] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Bootstrap starting.
I20260809 19:21:23.005800 21596 log.cc:824] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:23.005929 21598 tablet_bootstrap.cc:654] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.006273 21598 log.cc:824] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:23.006356 21599 tablet_bootstrap.cc:654] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.006575 21596 tablet_bootstrap.cc:492] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: No bootstrap required, opened a new log
I20260809 19:21:23.006632 21596 ts_tablet_manager.cc:1397] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:23.006664 21599 log.cc:824] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:23.006856 21598 tablet_bootstrap.cc:492] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: No bootstrap required, opened a new log
I20260809 19:21:23.006942 21598 ts_tablet_manager.cc:1397] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:23.007439 21599 tablet_bootstrap.cc:492] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: No bootstrap required, opened a new log
I20260809 19:21:23.007494 21599 ts_tablet_manager.cc:1397] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:23.008579 21598 raft_consensus.cc:348] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.008548 21596 raft_consensus.cc:348] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.008690 21598 raft_consensus.cc:374] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.008709 21598 raft_consensus.cc:729] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3411571940704655985281caf5335b63, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.008698 21596 raft_consensus.cc:374] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.008733 21596 raft_consensus.cc:729] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bade9033e47b420a92b19bc3d78ecf2c, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.008798 21598 consensus_queue.cc:260] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.008842 21596 consensus_queue.cc:260] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.009063 21598 ts_tablet_manager.cc:1428] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:21:23.009083 21596 ts_tablet_manager.cc:1428] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 19:21:23.009214 21553 heartbeater.cc:503] Master 127.9.91.126:39427 was elected leader, sending a full tablet report...
I20260809 19:21:23.009230 21112 heartbeater.cc:503] Master 127.9.91.126:39427 was elected leader, sending a full tablet report...
I20260809 19:21:23.009348 21599 raft_consensus.cc:348] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.009476 21599 raft_consensus.cc:374] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.009509 21599 raft_consensus.cc:729] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b94ca1d24542a0999fd4f9a18eafda, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.009793 21599 consensus_queue.cc:260] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.010052 21599 ts_tablet_manager.cc:1428] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:21:23.010090 21327 heartbeater.cc:503] Master 127.9.91.126:39427 was elected leader, sending a full tablet report...
W20260809 19:21:23.048768 21328 tablet.cc:2369] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:21:23.121984 21610 raft_consensus.cc:482] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:23.122095 21610 raft_consensus.cc:504] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.122421 21610 leader_election.cc:288] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317), 3411571940704655985281caf5335b63 (127.9.91.65:35717)
I20260809 19:21:23.125890 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.122571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41317 (local address 127.9.91.67:52685)
0809 19:21:23.122622 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.122632 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.122673 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.122785 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.122786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.122787 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:23.122852 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.122854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.123625 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.123626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.124864 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.124867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.125604 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.125605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.125705 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.125708 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.125760 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.125799 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":38}
I20260809 19:21:23.125888 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.122659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:36071 (local address 127.9.91.65:35717)
0809 19:21:23.122704 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.122706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.122852 (+ 146us) server_negotiation.cc:407] Connection header received
0809 19:21:23.122905 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.122906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.122941 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.122999 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:23.123121 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.123574 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.124965 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.125531 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.125606 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.125617 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.125778 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.125818 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:21:23.125888 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.122634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:52685 (local address 127.9.91.66:41317)
0809 19:21:23.122682 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.122684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.122694 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:23.122729 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.122730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.122740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.122782 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:23.122908 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.123550 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.124959 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.125523 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.125606 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.125617 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.125778 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.125818 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:21:23.125916 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.122598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35717 (local address 127.9.91.67:36071)
0809 19:21:23.122776 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.122786 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.122849 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.123000 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.123001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.123003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:23.123061 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.123062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.123658 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.123659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.124864 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.124868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.125605 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.125605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.125705 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.125708 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.125760 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.125798 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":15,"thread_start_us":123,"threads_started":1}
I20260809 19:21:23.126241 21038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6835da24886f4b018b1ae2a33cde6b61" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3411571940704655985281caf5335b63" is_pre_election: true
I20260809 19:21:23.126358 21038 raft_consensus.cc:2454] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 0.
I20260809 19:21:23.126560 21359 leader_election.cc:302] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, bade9033e47b420a92b19bc3d78ecf2c; no voters:
I20260809 19:21:23.126680 21610 raft_consensus.cc:2781] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:23.126655 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6835da24886f4b018b1ae2a33cde6b61" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda" is_pre_election: true
I20260809 19:21:23.126729 21610 raft_consensus.cc:482] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:23.126744 21610 raft_consensus.cc:3037] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.126794 21255 raft_consensus.cc:2454] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 0.
I20260809 19:21:23.127739 21610 raft_consensus.cc:504] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.127862 21610 leader_election.cc:288] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 election: Requested vote from peers 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317), 3411571940704655985281caf5335b63 (127.9.91.65:35717)
I20260809 19:21:23.127997 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6835da24886f4b018b1ae2a33cde6b61" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda"
I20260809 19:21:23.128026 21038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6835da24886f4b018b1ae2a33cde6b61" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3411571940704655985281caf5335b63"
I20260809 19:21:23.128074 21255 raft_consensus.cc:3037] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.128083 21038 raft_consensus.cc:3037] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.128890 21255 raft_consensus.cc:2454] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 1.
I20260809 19:21:23.128908 21038 raft_consensus.cc:2454] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 1.
W20260809 19:21:23.129056 21113 tablet.cc:2369] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:21:23.129086 21360 leader_election.cc:302] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66b94ca1d24542a0999fd4f9a18eafda, bade9033e47b420a92b19bc3d78ecf2c; no voters:
I20260809 19:21:23.129191 21610 raft_consensus.cc:2781] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:23.129333 21610 raft_consensus.cc:686] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 1 LEADER]: Becoming Leader. State: Replica: bade9033e47b420a92b19bc3d78ecf2c, State: Running, Role: LEADER
I20260809 19:21:23.129405 21610 consensus_queue.cc:237] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.130174 20794 catalog_manager.cc:5495] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c reported cstate change: term changed from 0 to 1, leader changed from <none> to bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67). New cstate: current_term: 1 leader_uuid: "bade9033e47b420a92b19bc3d78ecf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:23.144831 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.143003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.9.91.67:43541)
0809 19:21:23.143041 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.143043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.143050 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:23.143111 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.143111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.143126 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.143178 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.143255 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.143660 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.144311 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.144473 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.144512 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.144607 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.144697 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.144747 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.144762 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.144777 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:21:23.146445 21255 raft_consensus.cc:1261] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Refusing update from remote peer bade9033e47b420a92b19bc3d78ecf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.146445 21038 raft_consensus.cc:1261] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Refusing update from remote peer bade9033e47b420a92b19bc3d78ecf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.146732 21614 consensus_queue.cc:1035] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.146859 21614 consensus_queue.cc:1035] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.149355 21626 mvcc.cc:205] Tried to move back new op lower bound from 7316698247765381120 to 7316698247698595840. Current Snapshot: MvccSnapshot[applied={T|T < 7316698247765381120}]
I20260809 19:21:23.151103 20792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.151185 20792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.151813 20792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Sending DeleteTablet for 3 replicas of tablet 6835da24886f4b018b1ae2a33cde6b61
I20260809 19:21:23.152118 21001 tablet_service.cc:1514] Processing DeleteTablet for tablet 6835da24886f4b018b1ae2a33cde6b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50534
I20260809 19:21:23.152124 21438 tablet_service.cc:1514] Processing DeleteTablet for tablet 6835da24886f4b018b1ae2a33cde6b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:46016
I20260809 19:21:23.152284 21218 tablet_service.cc:1514] Processing DeleteTablet for tablet 6835da24886f4b018b1ae2a33cde6b61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:44888
I20260809 19:21:23.152488 21638 tablet_replica.cc:330] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: stopping tablet replica
I20260809 19:21:23.152539 21637 tablet_replica.cc:330] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: stopping tablet replica
I20260809 19:21:23.152643 21637 raft_consensus.cc:2229] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:23.152673 21638 raft_consensus.cc:2229] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.152751 21637 raft_consensus.cc:2258] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.152683 20792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:23.152812 20792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:23.153005 21638 raft_consensus.cc:2258] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.153340 21637 ts_tablet_manager.cc:1905] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.153555 21638 ts_tablet_manager.cc:1905] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.154129 21001 tablet_service.cc:1467] Processing CreateTablet for tablet e1a1b290c2a94fd59130727e803dda76 (DEFAULT_TABLE table=test-table [id=b2881303ff67437b86a9051e337e21b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.154263 21001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1a1b290c2a94fd59130727e803dda76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.155179 21637 ts_tablet_manager.cc:1918] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:23.155280 21637 log.cc:1196] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/6835da24886f4b018b1ae2a33cde6b61
I20260809 19:21:23.155412 21638 ts_tablet_manager.cc:1918] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:23.155525 21637 ts_tablet_manager.cc:1939] T 6835da24886f4b018b1ae2a33cde6b61 P bade9033e47b420a92b19bc3d78ecf2c: Deleting consensus metadata
I20260809 19:21:23.155531 21638 log.cc:1196] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/6835da24886f4b018b1ae2a33cde6b61
I20260809 19:21:23.155645 21598 tablet_bootstrap.cc:492] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: Bootstrap starting.
I20260809 19:21:23.155901 21638 ts_tablet_manager.cc:1939] T 6835da24886f4b018b1ae2a33cde6b61 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting consensus metadata
I20260809 19:21:23.156005 21598 tablet_bootstrap.cc:654] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.156457 20770 catalog_manager.cc:4841] TS 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317): tablet 6835da24886f4b018b1ae2a33cde6b61 (table test-table [id=0e19619d6e6e4442b9dd46301ad64f43]) successfully deleted
I20260809 19:21:23.156539 20770 catalog_manager.cc:4841] TS bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541): tablet 6835da24886f4b018b1ae2a33cde6b61 (table test-table [id=0e19619d6e6e4442b9dd46301ad64f43]) successfully deleted
I20260809 19:21:23.156541 21438 tablet_service.cc:1467] Processing CreateTablet for tablet e1a1b290c2a94fd59130727e803dda76 (DEFAULT_TABLE table=test-table [id=b2881303ff67437b86a9051e337e21b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.156628 21438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1a1b290c2a94fd59130727e803dda76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.156627 21598 tablet_bootstrap.cc:492] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: No bootstrap required, opened a new log
I20260809 19:21:23.156682 21598 ts_tablet_manager.cc:1397] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:23.156816 21598 raft_consensus.cc:348] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.156873 21598 raft_consensus.cc:374] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.156893 21598 raft_consensus.cc:729] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3411571940704655985281caf5335b63, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.156937 21598 consensus_queue.cc:260] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.157029 21598 ts_tablet_manager.cc:1428] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.157570 21596 tablet_bootstrap.cc:492] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Bootstrap starting.
I20260809 19:21:23.157615 21218 tablet_service.cc:1467] Processing CreateTablet for tablet e1a1b290c2a94fd59130727e803dda76 (DEFAULT_TABLE table=test-table [id=b2881303ff67437b86a9051e337e21b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.157719 21218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1a1b290c2a94fd59130727e803dda76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.157955 21596 tablet_bootstrap.cc:654] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.158607 21596 tablet_bootstrap.cc:492] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: No bootstrap required, opened a new log
I20260809 19:21:23.158648 21596 ts_tablet_manager.cc:1397] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:23.158779 21596 raft_consensus.cc:348] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.158856 21596 raft_consensus.cc:374] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.158859 21599 tablet_bootstrap.cc:492] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: Bootstrap starting.
I20260809 19:21:23.158877 21596 raft_consensus.cc:729] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bade9033e47b420a92b19bc3d78ecf2c, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.158941 21596 consensus_queue.cc:260] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.159027 21596 ts_tablet_manager.cc:1428] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.159247 21599 tablet_bootstrap.cc:654] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.160107 21599 tablet_bootstrap.cc:492] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: No bootstrap required, opened a new log
I20260809 19:21:23.160154 21599 ts_tablet_manager.cc:1397] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:23.160283 21599 raft_consensus.cc:348] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.160324 21599 raft_consensus.cc:374] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.160336 21599 raft_consensus.cc:729] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b94ca1d24542a0999fd4f9a18eafda, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.160367 21599 consensus_queue.cc:260] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.160439 21599 ts_tablet_manager.cc:1428] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.163014 21636 tablet_replica.cc:330] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: stopping tablet replica
I20260809 19:21:23.163108 21636 raft_consensus.cc:2229] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.163198 21636 raft_consensus.cc:2258] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.163522 21636 ts_tablet_manager.cc:1905] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.164660 21636 ts_tablet_manager.cc:1918] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:23.164733 21636 log.cc:1196] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/6835da24886f4b018b1ae2a33cde6b61
I20260809 19:21:23.165179 21636 ts_tablet_manager.cc:1939] T 6835da24886f4b018b1ae2a33cde6b61 P 3411571940704655985281caf5335b63: Deleting consensus metadata
I20260809 19:21:23.165501 20769 catalog_manager.cc:4841] TS 3411571940704655985281caf5335b63 (127.9.91.65:35717): tablet 6835da24886f4b018b1ae2a33cde6b61 (table test-table [id=0e19619d6e6e4442b9dd46301ad64f43]) successfully deleted
I20260809 19:21:23.170619 21612 raft_consensus.cc:482] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:23.170687 21612 raft_consensus.cc:504] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.170958 21612 leader_election.cc:288] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3411571940704655985281caf5335b63 (127.9.91.65:35717), bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541)
I20260809 19:21:23.174005 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.171130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:36431 (local address 127.9.91.65:35717)
0809 19:21:23.171184 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.171186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.171366 (+ 180us) server_negotiation.cc:407] Connection header received
0809 19:21:23.171456 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.171457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.171465 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.171490 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:23.171701 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.172309 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.173230 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.173684 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.173752 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.173761 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.173919 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.173950 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:21:23.174045 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.171056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35717 (local address 127.9.91.66:36431)
0809 19:21:23.171306 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.171315 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.171388 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.171541 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.171541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.171543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:23.171620 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.171621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.172378 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.172379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.173150 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.173152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.173778 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.173779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.173850 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.173854 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.173910 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.173946 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":141,"thread_start_us":40,"threads_started":1}
I20260809 19:21:23.174242 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.171056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:43541 (local address 127.9.91.66:49631)
0809 19:21:23.171155 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.171166 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.171205 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.171364 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.171364 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.171366 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:23.171421 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.171423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.172175 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.172176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.173337 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.173339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.173998 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.173999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.174083 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.174087 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.174137 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.174179 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":3}
I20260809 19:21:23.174306 21038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a1b290c2a94fd59130727e803dda76" candidate_uuid: "66b94ca1d24542a0999fd4f9a18eafda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3411571940704655985281caf5335b63" is_pre_election: true
I20260809 19:21:23.174373 21038 raft_consensus.cc:2454] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b94ca1d24542a0999fd4f9a18eafda in term 0.
I20260809 19:21:23.174463 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.171117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:49631 (local address 127.9.91.67:43541)
0809 19:21:23.171194 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.171196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.171204 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:23.171274 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.171274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.171282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.171327 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:23.171489 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.172097 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.173420 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.173901 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.173960 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.173967 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.174372 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.174409 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:21:23.174552 21150 leader_election.cc:302] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, 66b94ca1d24542a0999fd4f9a18eafda; no voters:
I20260809 19:21:23.174659 21612 raft_consensus.cc:2781] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:23.174695 21612 raft_consensus.cc:482] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:23.174638 21476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a1b290c2a94fd59130727e803dda76" candidate_uuid: "66b94ca1d24542a0999fd4f9a18eafda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade9033e47b420a92b19bc3d78ecf2c" is_pre_election: true
I20260809 19:21:23.174721 21612 raft_consensus.cc:3037] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.174738 21476 raft_consensus.cc:2454] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66b94ca1d24542a0999fd4f9a18eafda in term 0.
I20260809 19:21:23.175396 21612 raft_consensus.cc:504] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.175521 21612 leader_election.cc:288] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [CANDIDATE]: Term 1 election: Requested vote from peers 3411571940704655985281caf5335b63 (127.9.91.65:35717), bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541)
I20260809 19:21:23.175706 21038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a1b290c2a94fd59130727e803dda76" candidate_uuid: "66b94ca1d24542a0999fd4f9a18eafda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3411571940704655985281caf5335b63"
I20260809 19:21:23.175794 21038 raft_consensus.cc:3037] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.176354 21038 raft_consensus.cc:2454] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b94ca1d24542a0999fd4f9a18eafda in term 1.
I20260809 19:21:23.176473 21476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1a1b290c2a94fd59130727e803dda76" candidate_uuid: "66b94ca1d24542a0999fd4f9a18eafda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade9033e47b420a92b19bc3d78ecf2c"
I20260809 19:21:23.176508 21150 leader_election.cc:302] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, 66b94ca1d24542a0999fd4f9a18eafda; no voters:
I20260809 19:21:23.176534 21476 raft_consensus.cc:3037] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.176589 21612 raft_consensus.cc:2781] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:23.176733 21612 raft_consensus.cc:686] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 LEADER]: Becoming Leader. State: Replica: 66b94ca1d24542a0999fd4f9a18eafda, State: Running, Role: LEADER
I20260809 19:21:23.176790 21612 consensus_queue.cc:237] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.177210 21476 raft_consensus.cc:2454] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66b94ca1d24542a0999fd4f9a18eafda in term 1.
I20260809 19:21:23.177352 20794 catalog_manager.cc:5495] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda reported cstate change: term changed from 0 to 1, leader changed from <none> to 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66). New cstate: current_term: 1 leader_uuid: "66b94ca1d24542a0999fd4f9a18eafda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } health_report { overall_health: UNKNOWN } } }
W20260809 19:21:23.201838 21554 tablet.cc:2369] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303282, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.126@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:23.210412 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.203443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:39427 (local address 127.0.0.1:33820)
0809 19:21:23.203570 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.203580 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.203642 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.203763 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.203763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.204031 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:23.204072 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.204074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.204950 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.204951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.205449 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.205451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.205874 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.205874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.205909 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.206411 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:23.206415 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:23.207917 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:23.209514 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:23.209521 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:23.209523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:23.209698 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:23.209894 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:23.209895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:23.209895 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:23.209942 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:23.210191 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:23.210194 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:23.210230 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.210275 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.210349 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260809 19:21:23.210501 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.203501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.9.91.126:39427)
0809 19:21:23.203552 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.203553 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.203648 (+ 95us) server_negotiation.cc:407] Connection header received
0809 19:21:23.203693 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.203693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.203704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.203755 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:23.204141 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.204883 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.205518 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.205805 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.205870 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.207999 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:23.208001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:23.208001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:23.208008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:23.209440 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:23.209769 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:23.209770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:23.209770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:23.209834 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:23.209993 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:23.209994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:23.209994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:23.210126 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:23.210202 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.210324 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.210417 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":28}
I20260809 19:21:23.212283 20794 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:33820:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 19:21:23.212481 20794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:23.213689 21218 tablet_service.cc:1467] Processing CreateTablet for tablet 8b61833faeca446ab58188c1dfc45c54 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c09e7ad358e44e1284945e9c0bc47fa4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:21:23.213713 21001 tablet_service.cc:1467] Processing CreateTablet for tablet 8b61833faeca446ab58188c1dfc45c54 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c09e7ad358e44e1284945e9c0bc47fa4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:21:23.213830 21001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b61833faeca446ab58188c1dfc45c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.213830 21218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b61833faeca446ab58188c1dfc45c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.215194 21599 tablet_bootstrap.cc:492] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: Bootstrap starting.
I20260809 19:21:23.215252 21438 tablet_service.cc:1467] Processing CreateTablet for tablet 8b61833faeca446ab58188c1dfc45c54 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c09e7ad358e44e1284945e9c0bc47fa4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 19:21:23.215376 21438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b61833faeca446ab58188c1dfc45c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.215595 21599 tablet_bootstrap.cc:654] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.215943 21598 tablet_bootstrap.cc:492] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: Bootstrap starting.
I20260809 19:21:23.216332 21598 tablet_bootstrap.cc:654] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.216486 21596 tablet_bootstrap.cc:492] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: Bootstrap starting.
I20260809 19:21:23.216859 21599 tablet_bootstrap.cc:492] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: No bootstrap required, opened a new log
I20260809 19:21:23.216881 21596 tablet_bootstrap.cc:654] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.216904 21599 ts_tablet_manager.cc:1397] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:23.217000 21598 tablet_bootstrap.cc:492] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: No bootstrap required, opened a new log
I20260809 19:21:23.217041 21598 ts_tablet_manager.cc:1397] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:23.217036 21599 raft_consensus.cc:348] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.217093 21599 raft_consensus.cc:374] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.217113 21599 raft_consensus.cc:729] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b94ca1d24542a0999fd4f9a18eafda, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.217151 21599 consensus_queue.cc:260] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.217190 21598 raft_consensus.cc:348] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.217229 21599 ts_tablet_manager.cc:1428] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.217253 21598 raft_consensus.cc:374] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.217271 21598 raft_consensus.cc:729] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3411571940704655985281caf5335b63, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.217312 21598 consensus_queue.cc:260] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.217337 21612 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.217391 21598 ts_tablet_manager.cc:1428] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.217391 21612 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:21:23.217504 21609 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.217586 21609 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:21:23.218258 21596 tablet_bootstrap.cc:492] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: No bootstrap required, opened a new log
I20260809 19:21:23.218299 21596 ts_tablet_manager.cc:1397] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:21:23.218418 21596 raft_consensus.cc:348] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.218472 21596 raft_consensus.cc:374] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.218483 21596 raft_consensus.cc:729] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bade9033e47b420a92b19bc3d78ecf2c, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.218515 21596 consensus_queue.cc:260] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.218596 21596 ts_tablet_manager.cc:1428] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.218618 21614 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.218688 21614 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:21:23.247020 21609 raft_consensus.cc:482] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:23.247087 21609 raft_consensus.cc:504] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.247377 21609 leader_election.cc:288] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317), bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541)
I20260809 19:21:23.251228 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.247800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:40933 (local address 127.9.91.67:43541)
0809 19:21:23.247840 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.247842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.247945 (+ 103us) server_negotiation.cc:407] Connection header received
0809 19:21:23.248012 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.248013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.248021 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.248067 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:23.248204 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.248915 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.250294 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.250824 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.250897 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.250908 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.251114 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.251154 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:21:23.251227 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.247718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:43541 (local address 127.9.91.65:40933)
0809 19:21:23.247892 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.247902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.247964 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.248069 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.248070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.248071 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:23.248140 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.248142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.249013 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.249014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.250195 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.250198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.250943 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.250944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.251033 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.251038 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.251102 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.251143 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":77,"threads_started":1}
I20260809 19:21:23.251521 21476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b61833faeca446ab58188c1dfc45c54" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade9033e47b420a92b19bc3d78ecf2c" is_pre_election: true
I20260809 19:21:23.251619 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.247569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:43401 (local address 127.9.91.66:41317)
0809 19:21:23.247612 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.247614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.247831 (+ 217us) server_negotiation.cc:407] Connection header received
0809 19:21:23.247949 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.247951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.247961 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.248020 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:23.248261 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.248955 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.250002 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.251057 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.251094 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.251104 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.251521 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.251562 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:21:23.251614 21476 raft_consensus.cc:2454] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 0.
I20260809 19:21:23.251835 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.247521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41317 (local address 127.9.91.65:43401)
0809 19:21:23.247659 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.247670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.247865 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.248074 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.248075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.248077 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:23.248162 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.248164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.249037 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.249038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.249905 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.249908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.251161 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.251162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.251234 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.251238 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.251509 (+ 271us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.251550 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":40,"thread_start_us":80,"threads_started":1}
I20260809 19:21:23.251811 20935 leader_election.cc:302] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, bade9033e47b420a92b19bc3d78ecf2c; no voters:
I20260809 19:21:23.251947 21609 raft_consensus.cc:2781] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:23.251978 21609 raft_consensus.cc:482] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:23.251995 21609 raft_consensus.cc:3037] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.252089 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b61833faeca446ab58188c1dfc45c54" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda" is_pre_election: true
I20260809 19:21:23.252177 21255 raft_consensus.cc:2454] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 0.
I20260809 19:21:23.252725 21609 raft_consensus.cc:504] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.252836 21609 leader_election.cc:288] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 election: Requested vote from peers 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317), bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541)
I20260809 19:21:23.253041 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b61833faeca446ab58188c1dfc45c54" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda"
I20260809 19:21:23.253118 21255 raft_consensus.cc:3037] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.253091 21476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b61833faeca446ab58188c1dfc45c54" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade9033e47b420a92b19bc3d78ecf2c"
I20260809 19:21:23.253152 21476 raft_consensus.cc:3037] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.253716 21476 raft_consensus.cc:2454] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 1.
I20260809 19:21:23.253825 21255 raft_consensus.cc:2454] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 1.
I20260809 19:21:23.253849 20935 leader_election.cc:302] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, bade9033e47b420a92b19bc3d78ecf2c; no voters:
I20260809 19:21:23.253924 21609 raft_consensus.cc:2781] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:23.254056 21609 raft_consensus.cc:686] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [term 1 LEADER]: Becoming Leader. State: Replica: 3411571940704655985281caf5335b63, State: Running, Role: LEADER
I20260809 19:21:23.254122 21609 consensus_queue.cc:237] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } }
I20260809 19:21:23.254448 21671 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: TxnStatusTablet state changed. Reason: New leader 3411571940704655985281caf5335b63. Latest consensus state: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.254519 21671 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:21:23.254786 21674 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 19:21:23.254776 20794 catalog_manager.cc:5495] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3411571940704655985281caf5335b63 (127.9.91.65). New cstate: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } health_report { overall_health: HEALTHY } } }
I20260809 19:21:23.254891 21674 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 19:21:23.264612 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.261774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.9.91.65:35717)
0809 19:21:23.261809 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.261811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.261821 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:21:23.261852 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.261853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.261869 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.261936 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.262065 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.262733 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.263804 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.264057 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.264127 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.264313 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.264417 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.264478 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.264529 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.264546 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:21:23.264653 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.261664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35717 (local address 127.0.0.1:50540)
0809 19:21:23.261729 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.261741 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.261791 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.261950 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.261951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.261952 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:23.261996 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.261998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.262813 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.262814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.263714 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.263718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.264134 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.264135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.264211 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.264228 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.264477 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.264482 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.264527 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.264575 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:21:23.265625 21255 raft_consensus.cc:1261] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Refusing update from remote peer 3411571940704655985281caf5335b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.265671 21476 raft_consensus.cc:1261] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Refusing update from remote peer 3411571940704655985281caf5335b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.265849 21609 consensus_queue.cc:1035] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.265977 21609 consensus_queue.cc:1035] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.266392 21612 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: TxnStatusTablet state changed. Reason: New leader 3411571940704655985281caf5335b63. Latest consensus state: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.266466 21612 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:21:23.266513 21614 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: TxnStatusTablet state changed. Reason: New leader 3411571940704655985281caf5335b63. Latest consensus state: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.266585 21614 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:21:23.267453 21671 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.267529 21671 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:21:23.267593 21624 mvcc.cc:205] Tried to move back new op lower bound from 7316698248254464000 to 7316698248209334272. Current Snapshot: MvccSnapshot[applied={T|T < 7316698248254464000}]
I20260809 19:21:23.267935 21609 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.268007 21609 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P 3411571940704655985281caf5335b63: This TxnStatusTablet replica's current role is: LEADER
I20260809 19:21:23.268205 21612 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.268272 21612 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P 66b94ca1d24542a0999fd4f9a18eafda: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:21:23.268697 21620 mvcc.cc:205] Tried to move back new op lower bound from 7316698248254464000 to 7316698248209334272. Current Snapshot: MvccSnapshot[applied={T|T < 7316698248254464000}]
I20260809 19:21:23.268785 21614 tablet_replica.cc:439] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } } peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } }
I20260809 19:21:23.268844 21614 tablet_replica.cc:442] T 8b61833faeca446ab58188c1dfc45c54 P bade9033e47b420a92b19bc3d78ecf2c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 19:21:23.272861 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.269890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44892 (local address 127.9.91.66:41317)
0809 19:21:23.270001 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.270003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.270010 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:23.270044 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.270045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.270063 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.270124 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.270269 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.271006 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.272052 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.272346 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.272416 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.272565 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.272688 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.272744 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.272804 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.272819 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 19:21:23.277520 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.275235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:51943 (local address 127.9.91.65:35717)
0809 19:21:23.275280 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.275282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.275288 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:21:23.275303 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.275304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.275311 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.275351 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.275488 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.275895 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.276979 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.277157 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.277203 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.277366 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.277388 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.277415 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.277466 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.277479 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:21:23.277572 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.275030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35717 (local address 127.9.91.66:51943)
0809 19:21:23.275091 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.275102 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.275156 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.275371 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.275372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.275373 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:23.275430 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.275431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.276014 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.276015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.276906 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.276908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.277217 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.277218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.277280 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.277295 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.277418 (+ 123us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.277420 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.277465 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.277502 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:21:23.279493 21038 raft_consensus.cc:1261] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Refusing update from remote peer 66b94ca1d24542a0999fd4f9a18eafda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.279487 21476 raft_consensus.cc:1261] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Refusing update from remote peer 66b94ca1d24542a0999fd4f9a18eafda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.279768 21645 consensus_queue.cc:1035] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [LEADER]: Connected to new peer: Peer: permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.279903 21612 consensus_queue.cc:1035] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [LEADER]: Connected to new peer: Peer: permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.288154 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.285602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:57681 (local address 127.9.91.66:41317)
0809 19:21:23.285634 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.285635 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.285650 (+ 15us) server_negotiation.cc:407] Connection header received
0809 19:21:23.285668 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.285669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.285677 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.285723 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.285850 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.286289 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.287427 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.287715 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.287753 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.287962 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.287987 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.288019 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.288095 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.288110 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:23.288511 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.285264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41317 (local address 127.9.91.65:57681)
0809 19:21:23.285427 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.285437 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.285481 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.285732 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.285733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.285734 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:23.285786 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.285788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.286349 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.286350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.287329 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.287333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.287804 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.287805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.287868 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.287883 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.288044 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.288046 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.288092 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.288129 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260809 19:21:23.298331 20792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.298416 20792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.298971 20792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Sending DeleteTablet for 3 replicas of tablet e1a1b290c2a94fd59130727e803dda76
I20260809 19:21:23.299328 21001 tablet_service.cc:1514] Processing DeleteTablet for tablet e1a1b290c2a94fd59130727e803dda76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50534
I20260809 19:21:23.299419 21636 tablet_replica.cc:330] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: stopping tablet replica
I20260809 19:21:23.299468 21636 raft_consensus.cc:2229] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.299499 21636 raft_consensus.cc:2258] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.299556 21218 tablet_service.cc:1514] Processing DeleteTablet for tablet e1a1b290c2a94fd59130727e803dda76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:44888
I20260809 19:21:23.299619 21638 tablet_replica.cc:330] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: stopping tablet replica
I20260809 19:21:23.299649 21638 raft_consensus.cc:2229] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:23.299715 21638 raft_consensus.cc:2258] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.301591 21636 ts_tablet_manager.cc:1905] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.303117 21638 ts_tablet_manager.cc:1905] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.304612 21638 ts_tablet_manager.cc:1918] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:21:23.304674 21638 log.cc:1196] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/e1a1b290c2a94fd59130727e803dda76
I20260809 19:21:23.304881 21638 ts_tablet_manager.cc:1939] T e1a1b290c2a94fd59130727e803dda76 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting consensus metadata
I20260809 19:21:23.308451 21438 tablet_service.cc:1514] Processing DeleteTablet for tablet e1a1b290c2a94fd59130727e803dda76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:46016
I20260809 19:21:23.308559 21637 tablet_replica.cc:330] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: stopping tablet replica
I20260809 19:21:23.308624 21637 raft_consensus.cc:2229] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.308661 21637 raft_consensus.cc:2258] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.308979 21637 ts_tablet_manager.cc:1905] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.308995 21636 ts_tablet_manager.cc:1918] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:21:23.309038 21636 log.cc:1196] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/e1a1b290c2a94fd59130727e803dda76
I20260809 19:21:23.309254 21636 ts_tablet_manager.cc:1939] T e1a1b290c2a94fd59130727e803dda76 P 3411571940704655985281caf5335b63: Deleting consensus metadata
I20260809 19:21:23.309649 20770 catalog_manager.cc:4841] TS 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317): tablet e1a1b290c2a94fd59130727e803dda76 (table test-table [id=b2881303ff67437b86a9051e337e21b8]) successfully deleted
I20260809 19:21:23.309952 20769 catalog_manager.cc:4841] TS 3411571940704655985281caf5335b63 (127.9.91.65:35717): tablet e1a1b290c2a94fd59130727e803dda76 (table test-table [id=b2881303ff67437b86a9051e337e21b8]) successfully deleted
I20260809 19:21:23.310477 21637 ts_tablet_manager.cc:1918] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:21:23.310539 21637 log.cc:1196] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/e1a1b290c2a94fd59130727e803dda76
I20260809 19:21:23.310748 21637 ts_tablet_manager.cc:1939] T e1a1b290c2a94fd59130727e803dda76 P bade9033e47b420a92b19bc3d78ecf2c: Deleting consensus metadata
I20260809 19:21:23.311058 20770 catalog_manager.cc:4841] TS bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541): tablet e1a1b290c2a94fd59130727e803dda76 (table test-table [id=b2881303ff67437b86a9051e337e21b8]) successfully deleted
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[20728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:21:23.335261 20901 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:23.328160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33832 (local address 127.9.91.126:39427)
0809 19:21:23.328213 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.328215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.328503 (+ 288us) server_negotiation.cc:407] Connection header received
0809 19:21:23.328596 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.328597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.328609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.328636 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:23.329452 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.330212 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.331411 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.331709 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.331751 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.335198 (+ 3447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33832: BlockingRecv error: recv got EOF from 127.0.0.1:33832 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[20728](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM
I20260809 19:21:23.344393 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.336034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33834 (local address 127.9.91.126:39427)
0809 19:21:23.336069 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.336071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.336337 (+ 266us) server_negotiation.cc:407] Connection header received
0809 19:21:23.336387 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.336388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.336398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.336462 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:23.337009 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.337739 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.338684 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.339059 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.339125 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.341621 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:23.341623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:23.341623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:23.341631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:23.343215 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:23.343598 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:23.343600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:23.343600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:23.343666 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:23.343861 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:23.343862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:23.343862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:23.343997 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:23.344100 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.344224 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.344306 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:21:23.345927 20792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:23.346115 20792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:23.347579 21001 tablet_service.cc:1467] Processing CreateTablet for tablet fb17c0a8500c447989b28ddc7c898667 (DEFAULT_TABLE table=test-table [id=a5a4190db3814aa1adaa59a5cb22b26b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.347592 21438 tablet_service.cc:1467] Processing CreateTablet for tablet fb17c0a8500c447989b28ddc7c898667 (DEFAULT_TABLE table=test-table [id=a5a4190db3814aa1adaa59a5cb22b26b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.347718 21438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb17c0a8500c447989b28ddc7c898667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.347714 21001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb17c0a8500c447989b28ddc7c898667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.349109 21218 tablet_service.cc:1467] Processing CreateTablet for tablet fb17c0a8500c447989b28ddc7c898667 (DEFAULT_TABLE table=test-table [id=a5a4190db3814aa1adaa59a5cb22b26b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.349249 21218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb17c0a8500c447989b28ddc7c898667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.350564 21599 tablet_bootstrap.cc:492] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: Bootstrap starting.
I20260809 19:21:23.350579 21596 tablet_bootstrap.cc:492] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: Bootstrap starting.
I20260809 19:21:23.350906 21596 tablet_bootstrap.cc:654] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.351014 21599 tablet_bootstrap.cc:654] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.351202 21598 tablet_bootstrap.cc:492] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: Bootstrap starting.
I20260809 19:21:23.351572 21598 tablet_bootstrap.cc:654] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.351686 21599 tablet_bootstrap.cc:492] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: No bootstrap required, opened a new log
I20260809 19:21:23.351727 21599 ts_tablet_manager.cc:1397] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:23.351853 21599 raft_consensus.cc:348] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.351913 21599 raft_consensus.cc:374] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.351933 21599 raft_consensus.cc:729] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b94ca1d24542a0999fd4f9a18eafda, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.351974 21599 consensus_queue.cc:260] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.352059 21599 ts_tablet_manager.cc:1428] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.352965 21596 tablet_bootstrap.cc:492] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: No bootstrap required, opened a new log
I20260809 19:21:23.352993 21598 tablet_bootstrap.cc:492] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: No bootstrap required, opened a new log
I20260809 19:21:23.353015 21596 ts_tablet_manager.cc:1397] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:23.353037 21598 ts_tablet_manager.cc:1397] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:23.353123 21596 raft_consensus.cc:348] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.353202 21596 raft_consensus.cc:374] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.353169 21598 raft_consensus.cc:348] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.353225 21596 raft_consensus.cc:729] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bade9033e47b420a92b19bc3d78ecf2c, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.353228 21598 raft_consensus.cc:374] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.353247 21598 raft_consensus.cc:729] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3411571940704655985281caf5335b63, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.353268 21596 consensus_queue.cc:260] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.353288 21598 consensus_queue.cc:260] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.353354 21596 ts_tablet_manager.cc:1428] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:23.353382 21598 ts_tablet_manager.cc:1428] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.536512 21614 raft_consensus.cc:482] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:23.536603 21614 raft_consensus.cc:504] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.536764 21614 leader_election.cc:288] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3411571940704655985281caf5335b63 (127.9.91.65:35717), 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317)
I20260809 19:21:23.536954 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb17c0a8500c447989b28ddc7c898667" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda" is_pre_election: true
I20260809 19:21:23.536939 21038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb17c0a8500c447989b28ddc7c898667" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3411571940704655985281caf5335b63" is_pre_election: true
I20260809 19:21:23.537045 21255 raft_consensus.cc:2454] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 0.
I20260809 19:21:23.537045 21038 raft_consensus.cc:2454] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 0.
I20260809 19:21:23.537194 21359 leader_election.cc:302] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, bade9033e47b420a92b19bc3d78ecf2c; no voters:
I20260809 19:21:23.537328 21614 raft_consensus.cc:2781] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:23.537384 21614 raft_consensus.cc:482] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:23.537406 21614 raft_consensus.cc:3037] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.538137 21614 raft_consensus.cc:504] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.538280 21614 leader_election.cc:288] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 election: Requested vote from peers 3411571940704655985281caf5335b63 (127.9.91.65:35717), 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317)
I20260809 19:21:23.538430 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb17c0a8500c447989b28ddc7c898667" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda"
I20260809 19:21:23.538431 21038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb17c0a8500c447989b28ddc7c898667" candidate_uuid: "bade9033e47b420a92b19bc3d78ecf2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3411571940704655985281caf5335b63"
I20260809 19:21:23.538513 21038 raft_consensus.cc:3037] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.538513 21255 raft_consensus.cc:3037] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.539273 21255 raft_consensus.cc:2454] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 1.
I20260809 19:21:23.539273 21038 raft_consensus.cc:2454] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bade9033e47b420a92b19bc3d78ecf2c in term 1.
I20260809 19:21:23.539454 21359 leader_election.cc:302] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, bade9033e47b420a92b19bc3d78ecf2c; no voters:
I20260809 19:21:23.539587 21614 raft_consensus.cc:2781] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:23.539649 21614 raft_consensus.cc:686] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 1 LEADER]: Becoming Leader. State: Replica: bade9033e47b420a92b19bc3d78ecf2c, State: Running, Role: LEADER
I20260809 19:21:23.539700 21614 consensus_queue.cc:237] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.540246 20792 catalog_manager.cc:5495] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c reported cstate change: term changed from 0 to 1, leader changed from <none> to bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67). New cstate: current_term: 1 leader_uuid: "bade9033e47b420a92b19bc3d78ecf2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } health_report { overall_health: HEALTHY } } }
I20260809 19:21:23.578210 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.576261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.9.91.67:43541)
0809 19:21:23.576308 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.576310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.576319 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:23.576360 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.576360 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.576369 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.576409 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.576561 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.577045 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.577729 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.577903 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.577943 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.578045 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.578069 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.578098 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.578128 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.578145 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:21:23.578886 21038 raft_consensus.cc:1261] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Refusing update from remote peer bade9033e47b420a92b19bc3d78ecf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.578889 21255 raft_consensus.cc:1261] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Refusing update from remote peer bade9033e47b420a92b19bc3d78ecf2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.579097 21610 consensus_queue.cc:1035] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.579208 21614 consensus_queue.cc:1035] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.581254 20792 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.581327 20792 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.581830 20792 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Sending DeleteTablet for 3 replicas of tablet fb17c0a8500c447989b28ddc7c898667
I20260809 19:21:23.582041 21001 tablet_service.cc:1514] Processing DeleteTablet for tablet fb17c0a8500c447989b28ddc7c898667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50534
I20260809 19:21:23.582057 21218 tablet_service.cc:1514] Processing DeleteTablet for tablet fb17c0a8500c447989b28ddc7c898667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:44888
I20260809 19:21:23.582096 21438 tablet_service.cc:1514] Processing DeleteTablet for tablet fb17c0a8500c447989b28ddc7c898667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:46016
I20260809 19:21:23.582139 21636 tablet_replica.cc:330] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: stopping tablet replica
I20260809 19:21:23.582201 21636 raft_consensus.cc:2229] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.582238 21636 raft_consensus.cc:2258] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.582309 21638 tablet_replica.cc:330] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: stopping tablet replica
I20260809 19:21:23.582360 21638 raft_consensus.cc:2229] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.582394 21638 raft_consensus.cc:2258] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.582461 21637 tablet_replica.cc:330] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: stopping tablet replica
I20260809 19:21:23.582415 20792 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 19:21:23.582509 21637 raft_consensus.cc:2229] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 1 LEADER]: Raft consensus shutting down.
W20260809 19:21:23.582545 20792 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:23.582559 21637 raft_consensus.cc:2258] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.582947 21638 ts_tablet_manager.cc:1905] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.583051 21636 ts_tablet_manager.cc:1905] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.583773 21218 tablet_service.cc:1467] Processing CreateTablet for tablet 862d6402fc994ea9acda16bca192fb29 (DEFAULT_TABLE table=test-table [id=a18970a6543547bd93955ab6647600b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.583786 21438 tablet_service.cc:1467] Processing CreateTablet for tablet 862d6402fc994ea9acda16bca192fb29 (DEFAULT_TABLE table=test-table [id=a18970a6543547bd93955ab6647600b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.583889 21438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862d6402fc994ea9acda16bca192fb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.583889 21218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862d6402fc994ea9acda16bca192fb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.584416 21638 ts_tablet_manager.cc:1918] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:23.584466 21638 log.cc:1196] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/fb17c0a8500c447989b28ddc7c898667
I20260809 19:21:23.584457 21636 ts_tablet_manager.cc:1918] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:23.584504 21636 log.cc:1196] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/fb17c0a8500c447989b28ddc7c898667
I20260809 19:21:23.584679 21638 ts_tablet_manager.cc:1939] T fb17c0a8500c447989b28ddc7c898667 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting consensus metadata
I20260809 19:21:23.584715 21636 ts_tablet_manager.cc:1939] T fb17c0a8500c447989b28ddc7c898667 P 3411571940704655985281caf5335b63: Deleting consensus metadata
I20260809 19:21:23.585008 21599 tablet_bootstrap.cc:492] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: Bootstrap starting.
I20260809 19:21:23.585093 20770 catalog_manager.cc:4841] TS 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317): tablet fb17c0a8500c447989b28ddc7c898667 (table test-table [id=a5a4190db3814aa1adaa59a5cb22b26b]) successfully deleted
I20260809 19:21:23.585160 21596 tablet_bootstrap.cc:492] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: Bootstrap starting.
I20260809 19:21:23.585223 21637 ts_tablet_manager.cc:1905] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.585319 21599 tablet_bootstrap.cc:654] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.585582 21596 tablet_bootstrap.cc:654] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.585995 21599 tablet_bootstrap.cc:492] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: No bootstrap required, opened a new log
I20260809 19:21:23.586037 21599 ts_tablet_manager.cc:1397] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:23.586150 21599 raft_consensus.cc:348] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.586203 21599 raft_consensus.cc:374] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.586216 21599 raft_consensus.cc:729] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66b94ca1d24542a0999fd4f9a18eafda, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.586251 21599 consensus_queue.cc:260] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.586316 21599 ts_tablet_manager.cc:1428] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.586344 21637 ts_tablet_manager.cc:1918] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:23.586383 21637 log.cc:1196] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/fb17c0a8500c447989b28ddc7c898667
I20260809 19:21:23.586511 21637 ts_tablet_manager.cc:1939] T fb17c0a8500c447989b28ddc7c898667 P bade9033e47b420a92b19bc3d78ecf2c: Deleting consensus metadata
I20260809 19:21:23.586633 21596 tablet_bootstrap.cc:492] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: No bootstrap required, opened a new log
I20260809 19:21:23.586673 21596 ts_tablet_manager.cc:1397] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260809 19:21:23.586786 21596 raft_consensus.cc:348] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.586820 20769 catalog_manager.cc:4841] TS 3411571940704655985281caf5335b63 (127.9.91.65:35717): tablet fb17c0a8500c447989b28ddc7c898667 (table test-table [id=a5a4190db3814aa1adaa59a5cb22b26b]) successfully deleted
I20260809 19:21:23.586840 21596 raft_consensus.cc:374] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.586839 21001 tablet_service.cc:1467] Processing CreateTablet for tablet 862d6402fc994ea9acda16bca192fb29 (DEFAULT_TABLE table=test-table [id=a18970a6543547bd93955ab6647600b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:23.586860 21596 raft_consensus.cc:729] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bade9033e47b420a92b19bc3d78ecf2c, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.586896 21596 consensus_queue.cc:260] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.586946 21001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 862d6402fc994ea9acda16bca192fb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:23.587009 21596 ts_tablet_manager.cc:1428] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.587212 20770 catalog_manager.cc:4841] TS bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541): tablet fb17c0a8500c447989b28ddc7c898667 (table test-table [id=a5a4190db3814aa1adaa59a5cb22b26b]) successfully deleted
I20260809 19:21:23.588114 21598 tablet_bootstrap.cc:492] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63: Bootstrap starting.
I20260809 19:21:23.588402 21598 tablet_bootstrap.cc:654] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:23.588812 21598 tablet_bootstrap.cc:492] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63: No bootstrap required, opened a new log
I20260809 19:21:23.588850 21598 ts_tablet_manager.cc:1397] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 19:21:23.588946 21598 raft_consensus.cc:348] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.588996 21598 raft_consensus.cc:374] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:23.589012 21598 raft_consensus.cc:729] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3411571940704655985281caf5335b63, State: Initialized, Role: FOLLOWER
I20260809 19:21:23.589057 21598 consensus_queue.cc:260] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.589143 21598 ts_tablet_manager.cc:1428] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:23.735613 21609 raft_consensus.cc:482] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:23.735692 21609 raft_consensus.cc:504] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.735801 21609 leader_election.cc:288] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317), bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541)
I20260809 19:21:23.735991 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862d6402fc994ea9acda16bca192fb29" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda" is_pre_election: true
I20260809 19:21:23.736078 21255 raft_consensus.cc:2454] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 0.
I20260809 19:21:23.736050 21476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862d6402fc994ea9acda16bca192fb29" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade9033e47b420a92b19bc3d78ecf2c" is_pre_election: true
I20260809 19:21:23.736166 21476 raft_consensus.cc:2454] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 0.
I20260809 19:21:23.736227 20935 leader_election.cc:302] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, 66b94ca1d24542a0999fd4f9a18eafda; no voters:
I20260809 19:21:23.736341 21609 raft_consensus.cc:2781] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:23.736390 21609 raft_consensus.cc:482] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:23.736407 21609 raft_consensus.cc:3037] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.737013 21609 raft_consensus.cc:504] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.737113 21609 leader_election.cc:288] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 election: Requested vote from peers 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317), bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541)
I20260809 19:21:23.737274 21255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862d6402fc994ea9acda16bca192fb29" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66b94ca1d24542a0999fd4f9a18eafda"
I20260809 19:21:23.737301 21476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "862d6402fc994ea9acda16bca192fb29" candidate_uuid: "3411571940704655985281caf5335b63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bade9033e47b420a92b19bc3d78ecf2c"
I20260809 19:21:23.737370 21476 raft_consensus.cc:3037] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.737367 21255 raft_consensus.cc:3037] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:23.738162 21476 raft_consensus.cc:2454] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 1.
I20260809 19:21:23.738161 21255 raft_consensus.cc:2454] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3411571940704655985281caf5335b63 in term 1.
I20260809 19:21:23.738332 20935 leader_election.cc:302] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3411571940704655985281caf5335b63, 66b94ca1d24542a0999fd4f9a18eafda; no voters:
I20260809 19:21:23.738436 21609 raft_consensus.cc:2781] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:23.738507 21609 raft_consensus.cc:686] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 1 LEADER]: Becoming Leader. State: Replica: 3411571940704655985281caf5335b63, State: Running, Role: LEADER
I20260809 19:21:23.738564 21609 consensus_queue.cc:237] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } }
I20260809 19:21:23.739140 20794 catalog_manager.cc:5495] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3411571940704655985281caf5335b63 (127.9.91.65). New cstate: current_term: 1 leader_uuid: "3411571940704655985281caf5335b63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3411571940704655985281caf5335b63" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:23.767428 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.765569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.9.91.65:35717)
0809 19:21:23.765605 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.765607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.765614 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:23.765675 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.765675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.765683 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.765726 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.765807 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.766258 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.766893 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.767126 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.767175 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.767276 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.767302 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.767332 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.767363 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.767380 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:21:23.770869 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.769239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:45595 (local address 127.9.91.65:35717)
0809 19:21:23.769281 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:23.769282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:23.769289 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:23.769305 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:23.769306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:23.769312 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:23.769350 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:23.769416 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.769826 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.770432 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.770587 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.770626 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.770708 (+ 82us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.770728 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.770757 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:23.770787 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 19:21:23.770802 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":23}
I20260809 19:21:23.770877 21692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.769174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35717 (local address 127.9.91.65:45595)
0809 19:21:23.769216 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.769226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:23.769262 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:23.769352 (+ 90us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:23.769352 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:23.769353 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 19:21:23.769384 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.769385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.769870 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.769871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.770382 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:23.770383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:23.770630 (+ 247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:23.770631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:23.770663 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:23.770667 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 19:21:23.770758 (+ 91us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 19:21:23.770759 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:21:23.770785 (+ 26us) client_negotiation.cc:240] Negotiation successful
0809 19:21:23.770815 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":1}
I20260809 19:21:23.772509 21255 raft_consensus.cc:1261] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Refusing update from remote peer 3411571940704655985281caf5335b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.772509 21476 raft_consensus.cc:1261] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Refusing update from remote peer 3411571940704655985281caf5335b63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:23.772715 21609 consensus_queue.cc:1035] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66b94ca1d24542a0999fd4f9a18eafda" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.772814 21735 consensus_queue.cc:1035] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bade9033e47b420a92b19bc3d78ecf2c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 43541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:23.783903 20794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.783989 20794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:23.784578 20794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57925294b7194aaaa678d71180676650: Sending DeleteTablet for 3 replicas of tablet 862d6402fc994ea9acda16bca192fb29
I20260809 19:21:23.784793 21001 tablet_service.cc:1514] Processing DeleteTablet for tablet 862d6402fc994ea9acda16bca192fb29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50534
I20260809 19:21:23.784839 21218 tablet_service.cc:1514] Processing DeleteTablet for tablet 862d6402fc994ea9acda16bca192fb29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:44888
I20260809 19:21:23.784876 21636 tablet_replica.cc:330] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63: stopping tablet replica
I20260809 19:21:23.784919 21636 raft_consensus.cc:2229] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:23.784961 21638 tablet_replica.cc:330] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: stopping tablet replica
I20260809 19:21:23.784993 21636 raft_consensus.cc:2258] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.785008 21638 raft_consensus.cc:2229] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.785032 21638 raft_consensus.cc:2258] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.785295 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 20904
I20260809 19:21:23.785415 21636 ts_tablet_manager.cc:1905] T 862d6402fc994ea9acda16bca192fb29 P 3411571940704655985281caf5335b63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.792639 21438 tablet_service.cc:1514] Processing DeleteTablet for tablet 862d6402fc994ea9acda16bca192fb29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:23 UTC) from {username='oryx', principal='oryx/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:46016
I20260809 19:21:23.792732 21637 tablet_replica.cc:330] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: stopping tablet replica
I20260809 19:21:23.792781 21637 raft_consensus.cc:2229] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:23.792817 21637 raft_consensus.cc:2258] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:23.793094 21638 ts_tablet_manager.cc:1905] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.793203 21637 ts_tablet_manager.cc:1905] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:23.794517 21637 ts_tablet_manager.cc:1918] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:21:23.794534 21638 ts_tablet_manager.cc:1918] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 19:21:23.794570 21637 log.cc:1196] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/862d6402fc994ea9acda16bca192fb29
I20260809 19:21:23.794584 21638 log.cc:1196] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/862d6402fc994ea9acda16bca192fb29
I20260809 19:21:23.794770 21637 ts_tablet_manager.cc:1939] T 862d6402fc994ea9acda16bca192fb29 P bade9033e47b420a92b19bc3d78ecf2c: Deleting consensus metadata
I20260809 19:21:23.794787 21638 ts_tablet_manager.cc:1939] T 862d6402fc994ea9acda16bca192fb29 P 66b94ca1d24542a0999fd4f9a18eafda: Deleting consensus metadata
I20260809 19:21:23.795090 20770 catalog_manager.cc:4841] TS 66b94ca1d24542a0999fd4f9a18eafda (127.9.91.66:41317): tablet 862d6402fc994ea9acda16bca192fb29 (table test-table [id=a18970a6543547bd93955ab6647600b8]) successfully deleted
I20260809 19:21:23.795171 20770 catalog_manager.cc:4841] TS bade9033e47b420a92b19bc3d78ecf2c (127.9.91.67:43541): tablet 862d6402fc994ea9acda16bca192fb29 (table test-table [id=a18970a6543547bd93955ab6647600b8]) successfully deleted
W20260809 19:21:23.798801 20769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:35717 (error 108)
I20260809 19:21:23.799120 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 21125
I20260809 19:21:23.802461 21742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:23.799172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35717 (local address 127.0.0.1:50554)
0809 19:21:23.802371 (+ 3199us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:23.802428 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:35717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3181,"thread_start_us":40,"threads_started":1}
W20260809 19:21:23.802574 20769 catalog_manager.cc:4584] TS 3411571940704655985281caf5335b63 (127.9.91.65:35717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 862d6402fc994ea9acda16bca192fb29: Network error: Client connection negotiation failed: client connection to 127.9.91.65:35717: connect: Connection refused (error 111)
I20260809 19:21:23.806728 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 21337
I20260809 19:21:23.813011 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 20742
2026-08-09T19:21:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2732 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[21745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[21745](info): set up 2 sockets
Aug 09 19:21:23 dist-test-slave-1vmg krb5kdc[21745](info): commencing operation
krb5kdc: starting...
W20260809 19:21:24.936285 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:25 dist-test-slave-1vmg krb5kdc[21745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:25.043000 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:45837
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:45837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:25.120628 21771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:25.120816 21771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:25.120837 21771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:25.121939 21771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:25.121978 21771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:25.121991 21771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:25.122014 21771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:25.122037 21771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:25.123534 21771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:45837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:45837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:25.123754 21771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:25.123947 21771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:25.128309 21781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:25.128360 21771 server_base.cc:1034] running on GCE node
W20260809 19:21:25.128314 21779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:25.128289 21789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:25.128690 21771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:25.128970 21771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:25.130108 21771 hybrid_clock.cc:648] HybridClock initialized: now 1786303285130097 us; error 25 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T19:21:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1323 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:25 dist-test-slave-1vmg krb5kdc[21797](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:25 dist-test-slave-1vmg krb5kdc[21797](info): set up 2 sockets
Aug 09 19:21:25 dist-test-slave-1vmg krb5kdc[21797](info): commencing operation
krb5kdc: starting...
W20260809 19:21:26.260007 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:26.367465 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40861
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:40861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:26.445555 21818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:26.445746 21818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:26.445777 21818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:26.446879 21818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:26.446939 21818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:26.446961 21818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:26.446977 21818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:26.447000 21818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:26.448405 21818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:40861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:26.448627 21818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:26.448832 21818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:26.453222 21826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.453230 21829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.453284 21825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:26.453372 21818 server_base.cc:1034] running on GCE node
I20260809 19:21:26.453655 21818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:26.453887 21818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:26.455020 21818 hybrid_clock.cc:648] HybridClock initialized: now 1786303286455005 us; error 28 us; skew 500 ppm
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:26.460271 21818 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:26.460659 21818 webserver.cc:460] Webserver started at http://127.9.91.126:42755/ using document root <none> and password file <none>
I20260809 19:21:26.460843 21818 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:26.460888 21818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:26.461023 21818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:26.461966 21818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "f1a4ea77a0e047138e9a479e44aabc66"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "97658541aa9889464f0773a2f49d69e4"
server_key_iv: "ed03ef65977e64826e72c6aa12466c32"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.462287 21818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "f1a4ea77a0e047138e9a479e44aabc66"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "97658541aa9889464f0773a2f49d69e4"
server_key_iv: "ed03ef65977e64826e72c6aa12466c32"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.463533 21818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:26.464150 21836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.464296 21818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:26.464377 21818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "f1a4ea77a0e047138e9a479e44aabc66"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "97658541aa9889464f0773a2f49d69e4"
server_key_iv: "ed03ef65977e64826e72c6aa12466c32"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.464434 21818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:26.478125 21818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:26.479732 21818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:26.479866 21818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:26.483593 21818 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:40861
I20260809 19:21:26.483621 21911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:40861 every 8 connection(s)
I20260809 19:21:26.483906 21818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:26.484680 21914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:26.486737 21914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Bootstrap starting.
I20260809 19:21:26.487309 21914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:26.487565 21914 log.cc:824] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:26.488170 21914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: No bootstrap required, opened a new log
I20260809 19:21:26.489714 21914 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:26.489840 21914 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:26.489861 21914 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a4ea77a0e047138e9a479e44aabc66, State: Initialized, Role: FOLLOWER
I20260809 19:21:26.489998 21914 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:26.490067 21914 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:26.490089 21914 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:26.490132 21914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:26.490598 21914 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:26.490698 21914 leader_election.cc:302] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1a4ea77a0e047138e9a479e44aabc66; no voters:
I20260809 19:21:26.490857 21914 leader_election.cc:288] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:26.490898 21919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:26.491106 21914 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:26.491245 21919 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 LEADER]: Becoming Leader. State: Replica: f1a4ea77a0e047138e9a479e44aabc66, State: Running, Role: LEADER
I20260809 19:21:26.491348 21919 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:26.491402 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 21818
I20260809 19:21:26.491513 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:26.491720 9581 external_mini_cluster.cc:1428] Setting key bd4faf6b80b2a36c652d5988deb743ce
I20260809 19:21:26.491704 21920 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1a4ea77a0e047138e9a479e44aabc66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } } }
I20260809 19:21:26.491719 21921 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1a4ea77a0e047138e9a479e44aabc66. Latest consensus state: current_term: 1 leader_uuid: "f1a4ea77a0e047138e9a479e44aabc66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } } }
I20260809 19:21:26.491794 21920 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:26.491803 21921 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:26.492357 21935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:26.492939 21935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:26.494338 21935 catalog_manager.cc:1347] Generated new cluster ID: 82dd61b30cee4dc390b4355542c3fb66
I20260809 19:21:26.494395 21935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:26.501806 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:26.492266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.9.91.126:40861)
0809 19:21:26.492820 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:26.492828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:26.492852 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:21:26.492936 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:26.492941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:26.492980 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:26.493113 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:26.493794 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.495508 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.496241 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.496629 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.496720 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:26.498856 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:26.498861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:26.498861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:26.498872 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:26.500406 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.500726 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.500729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.500730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.500814 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.500987 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.500988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.500988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.501231 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:26.501372 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:26.501470 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 19:21:26.501599 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":74,"threads_started":1}
I20260809 19:21:26.507498 21935 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:26.508147 21935 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:26.517241 21935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Generated new TSK 0
I20260809 19:21:26.517391 21935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:21:26.547024 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40861
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:26.626156 21957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:26.626349 21957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:26.626371 21957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:26.627522 21957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:26.627562 21957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:26.627586 21957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:26.629010 21957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:26.629240 21957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:26.629442 21957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:26.634179 21964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.634223 21967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.634179 21971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:26.634545 21957 server_base.cc:1034] running on GCE node
I20260809 19:21:26.634722 21957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:26.634987 21957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:26.636130 21957 hybrid_clock.cc:648] HybridClock initialized: now 1786303286636115 us; error 34 us; skew 500 ppm
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:26.638530 21957 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:21:26.639001 21957 webserver.cc:460] Webserver started at http://127.9.91.65:46421/ using document root <none> and password file <none>
I20260809 19:21:26.639271 21957 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:26.639322 21957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:26.639451 21957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:26.640486 21957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "15502e23cd9b42b3aa16c2f41ce2f06a"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "99a3d9e055e6070f7e528e2322288c1e"
server_key_iv: "bde1fb09aacfe333b2853878f8f706d0"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.640830 21957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "15502e23cd9b42b3aa16c2f41ce2f06a"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "99a3d9e055e6070f7e528e2322288c1e"
server_key_iv: "bde1fb09aacfe333b2853878f8f706d0"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.642091 21957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:21:26.642740 21980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.642895 21957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.643004 21957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "15502e23cd9b42b3aa16c2f41ce2f06a"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "99a3d9e055e6070f7e528e2322288c1e"
server_key_iv: "bde1fb09aacfe333b2853878f8f706d0"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.643062 21957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:26.651799 21957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:26.653384 21957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:26.653508 21957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:26.653702 21957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:26.653976 21957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:26.654009 21957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.654035 21957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:26.654059 21957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.659595 21957 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:44199
I20260809 19:21:26.659622 22167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:44199 every 8 connection(s)
I20260809 19:21:26.659924 21957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:26.661320 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 21957
I20260809 19:21:26.661419 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:26.661623 9581 external_mini_cluster.cc:1428] Setting key b389f3ca7fcc2d255478a4090802a634
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:26.669847 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:26.660578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:53115 (local address 127.9.91.126:40861)
0809 19:21:26.660641 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:26.660643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:26.660802 (+ 159us) server_negotiation.cc:407] Connection header received
0809 19:21:26.661460 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:26.661461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:26.661473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:26.661534 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:26.662606 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.663305 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.664277 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.664582 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.664650 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:26.667063 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:26.667064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:26.667064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:26.667073 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:26.668424 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.668877 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.668878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.668879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.668931 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.669220 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.669222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.669222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.669343 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:26.669407 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:26.669708 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 19:21:26.669776 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:21:26.670070 22172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:26.660538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:53115)
0809 19:21:26.660689 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:26.660729 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:26.661325 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:26.661607 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:26.661610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:26.662322 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:26.662518 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:26.662523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.663395 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.663396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:26.664190 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:26.664193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.664664 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.664665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:26.664762 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:26.665241 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:26.665249 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:26.666977 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:26.668510 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:26.668519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:26.668521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:26.668805 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:26.668987 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:26.668988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:26.668988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:26.669120 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:26.669414 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:26.669419 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:26.669578 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 19:21:26.669694 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 19:21:26.669851 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260809 19:21:26.670547 22169 heartbeater.cc:347] Connected to a master server at 127.9.91.126:40861
I20260809 19:21:26.670641 22169 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
I20260809 19:21:26.670812 22169 heartbeater.cc:511] Master 127.9.91.126:40861 requested a full tablet report, sending...
I20260809 19:21:26.671307 21860 ts_manager.cc:194] Registered new tserver with Master: 15502e23cd9b42b3aa16c2f41ce2f06a (127.9.91.65:44199)
I20260809 19:21:26.671882 21860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:53115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:21:26.685294 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40861
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:26.765468 22176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:26.765669 22176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:26.765702 22176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:26.766863 22176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:26.766909 22176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:26.766976 22176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:26.768469 22176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:26.768703 22176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:26.768913 22176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:26.773588 22184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.773622 22186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.773656 22191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:26.773773 22176 server_base.cc:1034] running on GCE node
I20260809 19:21:26.774019 22176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:26.774259 22176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:26.775393 22176 hybrid_clock.cc:648] HybridClock initialized: now 1786303286775382 us; error 29 us; skew 500 ppm
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:26.777740 22176 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:21:26.778128 22176 webserver.cc:460] Webserver started at http://127.9.91.66:38951/ using document root <none> and password file <none>
I20260809 19:21:26.778312 22176 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:26.778350 22176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:26.778439 22176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:26.779417 22176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "a2add5df5be64e338dfd16d83ceca4d8"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "8dfc2a5b5a7b6c3b12d4044fe3b8e9e1"
server_key_iv: "236d3455142b8eec30edacddd06eba50"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.779704 22176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "a2add5df5be64e338dfd16d83ceca4d8"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "8dfc2a5b5a7b6c3b12d4044fe3b8e9e1"
server_key_iv: "236d3455142b8eec30edacddd06eba50"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.780850 22176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:26.781556 22198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.781713 22176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:26.781793 22176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "a2add5df5be64e338dfd16d83ceca4d8"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "8dfc2a5b5a7b6c3b12d4044fe3b8e9e1"
server_key_iv: "236d3455142b8eec30edacddd06eba50"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.781845 22176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:26.801136 22176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:26.803345 22176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:26.803467 22176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:26.803669 22176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:26.803942 22176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:26.803977 22176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.804003 22176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:26.804025 22176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.810287 22176 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:33921
I20260809 19:21:26.810324 22382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:33921 every 8 connection(s)
I20260809 19:21:26.810645 22176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:26.818864 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:26.811342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:60227 (local address 127.9.91.126:40861)
0809 19:21:26.811410 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:26.811411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:26.811570 (+ 159us) server_negotiation.cc:407] Connection header received
0809 19:21:26.812058 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:26.812059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:26.812072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:26.812152 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:26.813039 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.813554 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.814499 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.814712 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.814774 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:26.816663 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:26.816665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:26.816665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:26.816680 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:26.817890 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.818200 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.818200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.818201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.818247 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.818411 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.818412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.818412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.818502 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:26.818545 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:26.818736 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 19:21:26.818799 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:21:26.819048 22386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:26.811300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:60227)
0809 19:21:26.811462 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:26.811503 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:26.811961 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:26.812212 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:26.812216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:26.812757 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:26.812953 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:26.812957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.813626 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.813627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:26.814406 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:26.814408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.814779 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.814779 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:26.814868 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:26.815319 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:26.815327 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:26.816585 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:26.817952 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:26.817955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:26.817957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:26.818150 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:26.818288 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:26.818289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:26.818289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:26.818372 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:26.818541 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:26.818544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:26.818650 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 19:21:26.818720 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:21:26.818832 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260809 19:21:26.819551 22383 heartbeater.cc:347] Connected to a master server at 127.9.91.126:40861
I20260809 19:21:26.819635 22383 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:26.819715 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 22176
I20260809 19:21:26.819825 22383 heartbeater.cc:511] Master 127.9.91.126:40861 requested a full tablet report, sending...
I20260809 19:21:26.819837 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:26.820047 9581 external_mini_cluster.cc:1428] Setting key a7d600717051461138fe2e65c992c3cb
I20260809 19:21:26.820240 21860 ts_manager.cc:194] Registered new tserver with Master: a2add5df5be64e338dfd16d83ceca4d8 (127.9.91.66:33921)
I20260809 19:21:26.820838 21860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:60227
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:21:26.843562 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40861
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:26.923712 22391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:26.923890 22391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:26.923911 22391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:26.925020 22391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:26.925061 22391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:26.925086 22391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:26.926469 22391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:40861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:26.926693 22391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:26.926889 22391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:26.931515 22399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.931509 22397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:26.931510 22401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:26.931874 22391 server_base.cc:1034] running on GCE node
I20260809 19:21:26.932083 22391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:26.932299 22391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:26.933434 22391 hybrid_clock.cc:648] HybridClock initialized: now 1786303286933421 us; error 28 us; skew 500 ppm
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:26.935820 22391 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:21:26.936235 22391 webserver.cc:460] Webserver started at http://127.9.91.67:43985/ using document root <none> and password file <none>
I20260809 19:21:26.936440 22391 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:26.936487 22391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:26.936614 22391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:26.937537 22391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "5bf27e5c10a640929f44e54ee65bde59"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "30385d689890f9476b7387f117f55749"
server_key_iv: "0e4ee35cd288da690a569667a9e1ff5d"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.937868 22391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "5bf27e5c10a640929f44e54ee65bde59"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "30385d689890f9476b7387f117f55749"
server_key_iv: "0e4ee35cd288da690a569667a9e1ff5d"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.939157 22391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:26.939932 22408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.940099 22391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:26.940187 22391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "5bf27e5c10a640929f44e54ee65bde59"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "30385d689890f9476b7387f117f55749"
server_key_iv: "0e4ee35cd288da690a569667a9e1ff5d"
server_key_version: "encryptionkey@0"
I20260809 19:21:26.940254 22391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:26.948678 22391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:26.950335 22391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:26.950471 22391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:26.950676 22391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:26.951028 22391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:26.951071 22391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.951102 22391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:26.951125 22391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:26.957170 22391 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:42259
I20260809 19:21:26.957211 22597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:42259 every 8 connection(s)
I20260809 19:21:26.957500 22391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:26.958019 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 22391
I20260809 19:21:26.958137 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:26.958344 9581 external_mini_cluster.cc:1428] Setting key 1a127742b2bad36d4159addb3ddf7d63
Aug 09 19:21:26 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303286, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:26.966037 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:26.958152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:51595 (local address 127.9.91.126:40861)
0809 19:21:26.958232 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:26.958235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:26.958500 (+ 265us) server_negotiation.cc:407] Connection header received
0809 19:21:26.959110 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:26.959111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:26.959119 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:26.959152 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:26.960099 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.960590 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.961517 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.961692 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.961734 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:26.963567 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:26.963569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:26.963569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:26.963575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:26.964633 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.965033 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.965035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.965035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.965112 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:26.965353 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:26.965354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:26.965354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:26.965505 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:26.965569 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:26.965865 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 19:21:26.965948 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:21:26.966176 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:26.958130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:51595)
0809 19:21:26.958357 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:26.958406 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:26.959007 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:26.959228 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:26.959232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:26.959834 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:26.960016 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:26.960021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.960688 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.960689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:26.961455 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:26.961457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:26.961743 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:26.961744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:26.961834 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:26.962184 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:26.962191 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:26.963488 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:26.964718 (+ 1230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:26.964725 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:26.964727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:26.964968 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:26.965169 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:26.965170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:26.965170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:26.965290 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:26.965559 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:26.965566 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:26.965738 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 19:21:26.965840 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 19:21:26.965984 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260809 19:21:26.966586 22599 heartbeater.cc:347] Connected to a master server at 127.9.91.126:40861
I20260809 19:21:26.966683 22599 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:26.966849 22599 heartbeater.cc:511] Master 127.9.91.126:40861 requested a full tablet report, sending...
I20260809 19:21:26.967199 21860 ts_manager.cc:194] Registered new tserver with Master: 5bf27e5c10a640929f44e54ee65bde59 (127.9.91.67:42259)
I20260809 19:21:26.967563 21860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:51595
I20260809 19:21:26.969951 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab.
Entry for principal oryx/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab.
WARNING: no policy specified for oryx/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab.
Entry for principal oryx/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab.
WARNING: no policy specified for oryx/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab.
Entry for principal oryx/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab.
I20260809 19:21:27.032913 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 21957
I20260809 19:21:27.047741 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 22176
I20260809 19:21:27.057612 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 22391
I20260809 19:21:27.062711 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 21818
I20260809 19:21:27.071833 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40861
--webserver_interface=127.9.91.126
--webserver_port=42755
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:40861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:27.153307 22611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:27.153514 22611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:27.153549 22611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:27.154656 22611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:27.154701 22611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:27.154724 22611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:27.154743 22611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:27.154763 22611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:27.156247 22611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:40861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:40861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=42755
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:27.156498 22611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:27.156705 22611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:21:27.161356 22611 server_base.cc:1034] running on GCE node
W20260809 19:21:27.161309 22622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:27.161300 22625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:27.161309 22628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:27.161736 22611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:27.162019 22611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:27.163166 22611 hybrid_clock.cc:648] HybridClock initialized: now 1786303287163147 us; error 37 us; skew 500 ppm
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303287, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:27.165531 22611 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:27.165956 22611 webserver.cc:460] Webserver started at http://127.9.91.126:42755/ using document root <none> and password file <none>
I20260809 19:21:27.166175 22611 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:27.166244 22611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:27.167552 22611 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:27.168164 22635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.168313 22611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:27.168401 22611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "f1a4ea77a0e047138e9a479e44aabc66"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "97658541aa9889464f0773a2f49d69e4"
server_key_iv: "ed03ef65977e64826e72c6aa12466c32"
server_key_version: "encryptionkey@0"
I20260809 19:21:27.168697 22611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:27.189481 22611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:27.191128 22611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:27.191268 22611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:27.195325 22611 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:40861
I20260809 19:21:27.195353 22720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:40861 every 8 connection(s)
I20260809 19:21:27.195711 22611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:27.196260 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 22611
I20260809 19:21:27.196585 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:44199
--local_ip_for_outbound_sockets=127.9.91.65
--tserver_master_addrs=127.9.91.126:40861
--webserver_port=46421
--webserver_interface=127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:21:27.197413 22721 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:21:27.198194 22721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Bootstrap starting.
I20260809 19:21:27.206986 22721 log.cc:824] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:27.208520 22721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:21:27.208770 22721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Bootstrap complete.
I20260809 19:21:27.210827 22721 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:27.211084 22721 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a4ea77a0e047138e9a479e44aabc66, State: Initialized, Role: FOLLOWER
I20260809 19:21:27.211239 22721 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:27.211299 22721 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:27.211328 22721 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:27.211370 22721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:21:27.212281 22721 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:27.212373 22721 leader_election.cc:302] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1a4ea77a0e047138e9a479e44aabc66; no voters:
I20260809 19:21:27.212522 22721 leader_election.cc:288] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:21:27.212566 22728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:21:27.212690 22728 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [term 2 LEADER]: Becoming Leader. State: Replica: f1a4ea77a0e047138e9a479e44aabc66, State: Running, Role: LEADER
I20260809 19:21:27.212771 22721 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:27.212774 22728 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } }
I20260809 19:21:27.213068 22729 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f1a4ea77a0e047138e9a479e44aabc66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } } }
I20260809 19:21:27.213095 22728 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1a4ea77a0e047138e9a479e44aabc66. Latest consensus state: current_term: 2 leader_uuid: "f1a4ea77a0e047138e9a479e44aabc66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1a4ea77a0e047138e9a479e44aabc66" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 40861 } } }
I20260809 19:21:27.213264 22728 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:27.213200 22729 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:27.213551 22741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:27.214038 22741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:27.214179 22741 catalog_manager.cc:1259] Loaded cluster ID: 82dd61b30cee4dc390b4355542c3fb66
I20260809 19:21:27.214447 22741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:27.215785 22741 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:27.216050 22741 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f1a4ea77a0e047138e9a479e44aabc66: Loaded TSK: 0
I20260809 19:21:27.216325 22741 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:21:27.280606 22725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:27.280797 22725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:27.280831 22725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:27.281947 22725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:27.281993 22725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:27.282030 22725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:27.283540 22725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.65.keytab
--principal=oryx/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:44199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=46421
--tserver_master_addrs=127.9.91.126:40861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:27.283776 22725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:27.283977 22725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:27.289652 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:27.289678 22763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:27.289695 22766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:27.290215 22725 server_base.cc:1034] running on GCE node
I20260809 19:21:27.290422 22725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:27.290665 22725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:27.291810 22725 hybrid_clock.cc:648] HybridClock initialized: now 1786303287291796 us; error 33 us; skew 500 ppm
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303287, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:27.294492 22725 init.cc:377] Logged in from keytab as oryx/127.9.91.65@KRBTEST.COM (short username oryx)
I20260809 19:21:27.295059 22725 webserver.cc:460] Webserver started at http://127.9.91.65:46421/ using document root <none> and password file <none>
I20260809 19:21:27.295300 22725 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:27.295377 22725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:27.297071 22725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:27.297720 22773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.297883 22725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:27.297960 22725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "15502e23cd9b42b3aa16c2f41ce2f06a"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "99a3d9e055e6070f7e528e2322288c1e"
server_key_iv: "bde1fb09aacfe333b2853878f8f706d0"
server_key_version: "encryptionkey@0"
I20260809 19:21:27.298271 22725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:27.309691 22725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:27.311748 22725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:27.311857 22725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:27.312091 22725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:27.312440 22725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:27.312482 22725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.312512 22725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:27.312546 22725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.318902 22725 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:44199
I20260809 19:21:27.319006 22944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:44199 every 8 connection(s)
I20260809 19:21:27.319334 22725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:27.322086 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 22725
I20260809 19:21:27.322398 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:33921
--local_ip_for_outbound_sockets=127.9.91.66
--tserver_master_addrs=127.9.91.126:40861
--webserver_port=38951
--webserver_interface=127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:27.328469 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.320070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:45381)
0809 19:21:27.320267 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.320312 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.320860 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.321446 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.321450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.321967 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.322150 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.322154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.323429 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.323430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.324386 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.324389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.325227 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.325228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.325362 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.325839 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.325849 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.328294 (+ 2445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
W20260809 19:21:27.328660 22946 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:40861 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.91.126:40861: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
W20260809 19:21:27.338292 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.320175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:45381 (local address 127.9.91.126:40861)
0809 19:21:27.321059 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.321067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.321102 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:21:27.321215 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.321220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.321271 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.321405 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.322280 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.323330 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.324486 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.325117 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.329278 (+ 4161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.329452 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:45381: BlockingRecv error: recv got EOF from 127.9.91.65:45381 (error 108)
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":74,"threads_started":1}
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:27.339733 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.328848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:37573)
0809 19:21:27.329133 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.329142 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.329181 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.335127 (+ 5946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.335128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.335278 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.335329 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.335331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.336237 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.336238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.336996 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.336999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.337466 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.337468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.337533 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.337897 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.337902 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.339667 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":70,"threads_started":1}
W20260809 19:21:27.342824 22954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.329010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:37573 (local address 127.9.91.126:40861)
0809 19:21:27.334982 (+ 5972us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.334983 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.334992 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:27.335014 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.335015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.335025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.335395 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.335427 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.336131 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.337093 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.337390 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.337669 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.342761 (+ 5092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:37573: BlockingRecv error: recv got EOF from 127.9.91.65:37573 (error 108)
Metrics: {"server-negotiator.queue_time_us":5950,"thread_start_us":74,"threads_started":1}
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:21:27.345739 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.340027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:60539 (local address 127.9.91.126:40861)
0809 19:21:27.340058 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.340060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.340179 (+ 119us) server_negotiation.cc:407] Connection header received
0809 19:21:27.340198 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.340199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.340209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.340248 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.340506 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.341207 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.342139 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.342441 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.342488 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.345694 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:60539: BlockingRecv error: recv got EOF from 127.9.91.65:60539 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:21:27.345955 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.339952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:60539)
0809 19:21:27.340085 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.340093 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.340126 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.340281 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.340282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.340415 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.340445 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.340446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.341284 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.341286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.342057 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.342059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.342531 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.342532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.342581 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.343222 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.343227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.345577 (+ 2350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124}
W20260809 19:21:27.346063 22946 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:21:27.411660 22952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:27.411844 22952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:27.411866 22952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:27.413017 22952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:27.413055 22952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:27.413081 22952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:27.414546 22952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.66.keytab
--principal=oryx/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:33921
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=38951
--tserver_master_addrs=127.9.91.126:40861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:27.414724 22952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:27.414903 22952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:27.419534 22969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:27.419685 22952 server_base.cc:1034] running on GCE node
W20260809 19:21:27.419545 22964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:27.419535 22965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:27.420023 22952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:27.420251 22952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:27.421392 22952 hybrid_clock.cc:648] HybridClock initialized: now 1786303287421381 us; error 33 us; skew 500 ppm
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303287, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:27.423813 22952 init.cc:377] Logged in from keytab as oryx/127.9.91.66@KRBTEST.COM (short username oryx)
I20260809 19:21:27.424250 22952 webserver.cc:460] Webserver started at http://127.9.91.66:38951/ using document root <none> and password file <none>
I20260809 19:21:27.424443 22952 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:27.424512 22952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:27.425772 22952 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:27.426304 22979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.426458 22952 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:27.426545 22952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "a2add5df5be64e338dfd16d83ceca4d8"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "8dfc2a5b5a7b6c3b12d4044fe3b8e9e1"
server_key_iv: "236d3455142b8eec30edacddd06eba50"
server_key_version: "encryptionkey@0"
I20260809 19:21:27.426836 22952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:27.433676 22952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:27.435335 22952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:27.435470 22952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:27.435676 22952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:27.435951 22952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:27.435995 22952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.436028 22952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:27.436055 22952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.442185 22952 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:33921
I20260809 19:21:27.442231 23143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:33921 every 8 connection(s)
I20260809 19:21:27.442549 22952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:27.448251 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 22952
I20260809 19:21:27.448549 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:42259
--local_ip_for_outbound_sockets=127.9.91.67
--tserver_master_addrs=127.9.91.126:40861
--webserver_port=43985
--webserver_interface=127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:21:27.449136 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.443196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:48415)
0809 19:21:27.443349 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.443386 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.443815 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.444010 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.444013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.444526 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.444700 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.444705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.445305 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.445306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.446086 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.446088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.446389 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.446389 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.446482 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.446856 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.446863 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.448951 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
W20260809 19:21:27.449188 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.443224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:48415 (local address 127.9.91.126:40861)
0809 19:21:27.443280 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.443281 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.443464 (+ 183us) server_negotiation.cc:407] Connection header received
0809 19:21:27.443888 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.443889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.443901 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.443945 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.444769 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.445242 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.446155 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.446335 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.446374 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.449119 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:48415: BlockingRecv error: recv got EOF from 127.9.91.66:48415 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 19:21:27.450340 23144 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:40861 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.91.126:40861: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:27.454780 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.450590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:44851)
0809 19:21:27.450651 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.450660 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.450697 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.450828 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.450829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.450978 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.451024 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.451026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.451805 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.451806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.452283 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.452285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.452743 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.452744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.452793 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.453152 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.453156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.454739 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 19:21:27.454903 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.450652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:44851 (local address 127.9.91.126:40861)
0809 19:21:27.450702 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.450704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.450736 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:21:27.450759 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.450760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.450772 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.450819 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.451092 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.451706 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.452375 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.452654 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.452728 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.454845 (+ 2117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:44851: BlockingRecv error: recv got EOF from 127.9.91.66:44851 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:27.459044 23146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.455147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:39755)
0809 19:21:27.455188 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.455197 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.455234 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.455376 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.455376 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.455473 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.455496 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.455497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.456275 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.456275 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.456723 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.456724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.457115 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.457116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.457146 (+ 30us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.457491 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.457494 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.459005 (+ 1511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:21:27.459137 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.455235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:39755 (local address 127.9.91.126:40861)
0809 19:21:27.455270 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.455272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.455280 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:27.455301 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.455302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.455311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.455373 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.455555 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.456208 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.456789 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.457052 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.457122 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.459087 (+ 1965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:39755: BlockingRecv error: recv got EOF from 127.9.91.66:39755 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:21:27.459209 23144 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:21:27.531484 23147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:27.531670 23147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:27.531692 23147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:27.532753 23147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:27.532783 23147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:27.532807 23147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:27.534157 23147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/krb5kdc/oryx_127.9.91.67.keytab
--principal=oryx/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:42259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=43985
--tserver_master_addrs=127.9.91.126:40861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:27.534330 23147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:27.534503 23147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:27.539156 23156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:27.539197 23158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:27.539247 23160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:27.539345 23147 server_base.cc:1034] running on GCE node
I20260809 19:21:27.539592 23147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:27.539822 23147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:27.540946 23147 hybrid_clock.cc:648] HybridClock initialized: now 1786303287540936 us; error 26 us; skew 500 ppm
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303287, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:27.543265 23147 init.cc:377] Logged in from keytab as oryx/127.9.91.67@KRBTEST.COM (short username oryx)
I20260809 19:21:27.543691 23147 webserver.cc:460] Webserver started at http://127.9.91.67:43985/ using document root <none> and password file <none>
I20260809 19:21:27.543897 23147 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:27.543962 23147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:27.545225 23147 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:27.545761 23171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.545912 23147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.545992 23147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "5bf27e5c10a640929f44e54ee65bde59"
format_stamp: "Formatted at 2026-08-09 19:21:26 on dist-test-slave-1vmg"
server_key: "30385d689890f9476b7387f117f55749"
server_key_iv: "0e4ee35cd288da690a569667a9e1ff5d"
server_key_version: "encryptionkey@0"
I20260809 19:21:27.546290 23147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:27.554211 23147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:27.555836 23147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:27.555959 23147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:27.556151 23147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:27.556414 23147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:27.556447 23147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.556471 23147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:27.556496 23147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:27.562186 23147 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:42259
I20260809 19:21:27.562240 23340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:42259 every 8 connection(s)
I20260809 19:21:27.562529 23147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:27.563946 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 23147
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 19:21:27.569811 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.563197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:41201 (local address 127.9.91.126:40861)
0809 19:21:27.563242 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.563244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.563430 (+ 186us) server_negotiation.cc:407] Connection header received
0809 19:21:27.563935 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.563937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.563949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.564030 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.565104 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.565593 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.566513 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.566701 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.566755 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.569730 (+ 2975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:41201: BlockingRecv error: recv got EOF from 127.9.91.67:41201 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:21:27.569851 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.563165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:41201)
0809 19:21:27.563330 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.563360 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.563806 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.564162 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.564166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.564842 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.565015 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.565019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.565671 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.565672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.566436 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.566438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.566769 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.566770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.566894 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.567335 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.567344 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.569662 (+ 2318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
W20260809 19:21:27.570261 23342 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:40861 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.91.126:40861: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
I20260809 19:21:27.571544 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.564469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39328 (local address 127.9.91.126:40861)
0809 19:21:27.564520 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.564522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.564612 (+ 90us) server_negotiation.cc:407] Connection header received
0809 19:21:27.564677 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.564678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.564691 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.564741 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.565356 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.565841 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.566620 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.566825 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.566881 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.568134 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:27.568139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:27.568140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:27.568151 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:27.570044 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:27.570411 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:27.570414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:27.570415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:27.570560 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:27.570762 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:27.570762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:27.570763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:27.571047 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:27.571200 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:27.571323 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 19:21:27.571440 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:27.574313 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.570497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:43987)
0809 19:21:27.570534 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.570544 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.570587 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.570702 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.570703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.570809 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.570846 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.570847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.571501 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.571502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.572216 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.572218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.572585 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.572585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.572635 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.572910 (+ 275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.572913 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.574274 (+ 1361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 19:21:27.574419 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.570542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:43987 (local address 127.9.91.126:40861)
0809 19:21:27.570584 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.570586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.570593 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:27.570641 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.570641 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.570650 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.570689 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.570907 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.571418 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.572269 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.572525 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.572582 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.574363 (+ 1781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:43987: BlockingRecv error: recv got EOF from 127.9.91.67:43987 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:27 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:27.578357 23345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:27.574528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:52341)
0809 19:21:27.574615 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:27.574626 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:27.574676 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:27.574804 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:27.574805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:27.574964 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:27.574995 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.574997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.575582 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.575583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.576050 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:27.576052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.576380 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.576381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:27.576433 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.576800 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:27.576805 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:27.578296 (+ 1491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 19:21:27.578430 22949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:27.574572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:52341 (local address 127.9.91.126:40861)
0809 19:21:27.574602 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:27.574604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:27.574662 (+ 58us) server_negotiation.cc:407] Connection header received
0809 19:21:27.574734 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:27.574734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:27.574743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:27.574799 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:27.575076 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.575501 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.576124 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:27.576310 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:27.576361 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:27.578384 (+ 2023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:52341: BlockingRecv error: recv got EOF from 127.9.91.67:52341 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:21:27.578486 23342 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 19:21:28 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:28 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:28.351317 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:28.346430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:34795)
0809 19:21:28.346552 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:28.346562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:28.346615 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:28.346835 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:28.346837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:28.347069 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:28.347131 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:28.347133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.348059 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.348060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:28.348650 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:28.348652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.348956 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.348957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:28.348991 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:28.349442 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:28.349447 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:28.351225 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
W20260809 19:21:28.351389 23350 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:28.346498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:34795 (local address 127.9.91.126:40861)
0809 19:21:28.346615 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:28.346617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:28.346625 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:28.346722 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:28.346723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:28.346736 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:28.346812 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:28.347227 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.347959 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.348714 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.348904 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.348954 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:28.351326 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:34795: BlockingRecv error: recv got EOF from 127.9.91.65:34795 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
Aug 09 19:21:28 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:28 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:28.463817 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:28.459536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:45015)
0809 19:21:28.459663 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:28.459672 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:28.459726 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:28.459852 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:28.459853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:28.460041 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:28.460101 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:28.460103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.460709 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.460710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:28.461285 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:28.461286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.461583 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.461584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:28.461624 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:28.462092 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:28.462096 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:28.463750 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
W20260809 19:21:28.463909 23350 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:28.459629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:45015 (local address 127.9.91.126:40861)
0809 19:21:28.459683 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:28.459685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:28.459714 (+ 29us) server_negotiation.cc:407] Connection header received
0809 19:21:28.459790 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:28.459791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:28.459802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:28.459839 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:28.460146 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.460654 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.461345 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.461533 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.461576 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:28.463848 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:45015: BlockingRecv error: recv got EOF from 127.9.91.66:45015 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 19:21:28 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:28 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:28.583179 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:28.578824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:52091)
0809 19:21:28.578992 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:28.579001 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:28.579062 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:28.579181 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:28.579182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:28.579339 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:28.579389 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:28.579390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.580008 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.580009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:28.580542 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:28.580543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.580828 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.580829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:28.580865 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:28.581261 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:28.581265 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:28.583112 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
W20260809 19:21:28.583235 23350 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:28.578899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:52091 (local address 127.9.91.126:40861)
0809 19:21:28.578987 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:28.578989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:28.579055 (+ 66us) server_negotiation.cc:407] Connection header received
0809 19:21:28.579105 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:28.579106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:28.579116 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:28.579168 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:28.579436 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.579951 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.580602 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:28.580779 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:28.580826 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:28.583187 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:52091: BlockingRecv error: recv got EOF from 127.9.91.67:52091 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 19:21:29 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:29 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:29.355919 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:29.351713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:57253)
0809 19:21:29.351842 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:29.351852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:29.351902 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:29.352023 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:29.352024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:29.352188 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:29.352238 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:29.352240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.352829 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.352830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:29.353349 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:29.353350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.353711 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.353713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:29.353775 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:29.354220 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:29.354225 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:29.355853 (+ 1628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
W20260809 19:21:29.356014 23355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:29.351783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:57253 (local address 127.9.91.126:40861)
0809 19:21:29.351893 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:29.351894 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:29.351903 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:29.351951 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:29.351952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:29.351963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:29.352013 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:29.352288 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.352777 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.353412 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.353639 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.353693 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:29.355942 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:57253: BlockingRecv error: recv got EOF from 127.9.91.65:57253 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
Aug 09 19:21:29 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:29 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:29.468505 23356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:29.464270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:54177)
0809 19:21:29.464417 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:29.464429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:29.464498 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:29.464619 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:29.464620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:29.464787 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:29.464838 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:29.464839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.465417 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.465418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:29.465938 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:29.465939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.466204 (+ 265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.466205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:29.466242 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:29.466694 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:29.466699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:29.468441 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
W20260809 19:21:29.468598 23355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:29.464342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:54177 (local address 127.9.91.126:40861)
0809 19:21:29.464410 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:29.464412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:29.464485 (+ 73us) server_negotiation.cc:407] Connection header received
0809 19:21:29.464555 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:29.464556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:29.464567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:29.464603 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:29.464886 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.465369 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.465990 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.466158 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.466199 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:29.468530 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:54177: BlockingRecv error: recv got EOF from 127.9.91.66:54177 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 19:21:29 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:29 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:29.588011 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:29.583634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:45997)
0809 19:21:29.583778 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:29.583793 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:29.583867 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:29.584003 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:29.584004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:29.584180 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:29.584230 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:29.584231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.584853 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.584854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:29.585376 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:29.585378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.585705 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.585707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:29.585763 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:29.586254 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:29.586259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:29.587942 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
W20260809 19:21:29.588099 23355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:29.583750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:45997 (local address 127.9.91.126:40861)
0809 19:21:29.583804 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:29.583807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:29.583854 (+ 47us) server_negotiation.cc:407] Connection header received
0809 19:21:29.583924 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:29.583924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:29.583939 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:29.583987 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:29.584300 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.584780 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.585449 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:29.585640 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:29.585696 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:29.588026 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:45997: BlockingRecv error: recv got EOF from 127.9.91.67:45997 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 19:21:30 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:30 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:30.360754 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:30.356323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:45827)
0809 19:21:30.356469 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:30.356480 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:30.356539 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:30.356658 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:30.356659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:30.356823 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:30.356872 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:30.356874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.357466 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.357466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:30.358148 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:30.358152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.358494 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.358495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:30.358532 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:30.358985 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:30.358990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:30.360684 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
W20260809 19:21:30.360846 23361 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:30.356388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:45827 (local address 127.9.91.126:40861)
0809 19:21:30.356493 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:30.356495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:30.356537 (+ 42us) server_negotiation.cc:407] Connection header received
0809 19:21:30.356579 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:30.356579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:30.356591 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:30.356647 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:30.356921 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.357416 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.358217 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.358439 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.358491 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:30.360775 (+ 2284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:45827: BlockingRecv error: recv got EOF from 127.9.91.65:45827 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
Aug 09 19:21:30 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:30 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:30.473021 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:30.468901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:55723)
0809 19:21:30.469019 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:30.469030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:30.469089 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:30.469196 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:30.469197 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:30.469335 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:30.469378 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:30.469380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.469945 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.469946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:30.470468 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:30.470470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.470742 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.470743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:30.470778 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:30.471244 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:30.471250 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:30.472955 (+ 1705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
W20260809 19:21:30.473099 23361 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:30.468965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:55723 (local address 127.9.91.126:40861)
0809 19:21:30.469032 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:30.469034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:30.469074 (+ 40us) server_negotiation.cc:407] Connection header received
0809 19:21:30.469137 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:30.469138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:30.469147 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:30.469187 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:30.469428 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.469895 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.470523 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.470697 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.470739 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:30.473042 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:55723: BlockingRecv error: recv got EOF from 127.9.91.66:55723 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 19:21:30 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:30 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:30.592525 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:30.588386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:58955)
0809 19:21:30.588534 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:30.588543 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:30.588596 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:30.588706 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:30.588707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:30.588850 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:30.588900 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:30.588902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.589493 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.589494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:30.590023 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:30.590025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.590344 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.590344 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:30.590381 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:30.590764 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:30.590767 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:30.592459 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
W20260809 19:21:30.592600 23361 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:30.588455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:58955 (local address 127.9.91.126:40861)
0809 19:21:30.588506 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:30.588508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:30.588590 (+ 82us) server_negotiation.cc:407] Connection header received
0809 19:21:30.588646 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:30.588646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:30.588657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:30.588694 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:30.588946 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.589414 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.590098 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:30.590288 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:30.590337 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:30.592541 (+ 2204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:58955: BlockingRecv error: recv got EOF from 127.9.91.67:58955 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 19:21:31 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:31 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:31.366560 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:31.361196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:51073)
0809 19:21:31.361308 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:31.361318 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:31.361373 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:31.361622 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:31.361623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:31.361865 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:31.361933 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:31.361937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.362767 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.362768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:31.363500 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:31.363504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.363932 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.363934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:31.364020 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:31.364484 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:31.364490 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:31.366488 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":64,"threads_started":1}
W20260809 19:21:31.366681 23366 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:31.361367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:51073 (local address 127.9.91.126:40861)
0809 19:21:31.361476 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:31.361478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:31.361486 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:31.361507 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:31.361508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:31.361520 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:31.361584 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:31.362030 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.362688 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.363570 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.363853 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.363929 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:31.366611 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:51073: BlockingRecv error: recv got EOF from 127.9.91.65:51073 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
Aug 09 19:21:31 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:31 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:31.480180 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:31.473401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:43767)
0809 19:21:31.473525 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:31.473537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:31.473601 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:31.473722 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:31.473723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:31.473884 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:31.473933 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:31.473935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.474520 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.474521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:31.475092 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:31.475094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.475374 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.475375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:31.475413 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:31.478478 (+ 3065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:31.478482 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:31.480123 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
W20260809 19:21:31.480275 23366 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:31.473479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:43767 (local address 127.9.91.126:40861)
0809 19:21:31.473552 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:31.473554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:31.473591 (+ 37us) server_negotiation.cc:407] Connection header received
0809 19:21:31.473659 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:31.473660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:31.473670 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:31.473705 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:31.473987 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.474461 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.475149 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.475325 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.475368 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:31.480220 (+ 4852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:43767: BlockingRecv error: recv got EOF from 127.9.91.66:43767 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 19:21:31 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:31 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:31.597011 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:31.593034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:44417)
0809 19:21:31.593150 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:31.593160 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:31.593217 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:31.593328 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:31.593329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:31.593469 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:31.593526 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:31.593528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.594087 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.594088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:31.594602 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:31.594604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.594878 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.594878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:31.594943 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:31.595314 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:31.595318 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:31.596941 (+ 1623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
W20260809 19:21:31.597069 23366 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:31.593077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:44417 (local address 127.9.91.126:40861)
0809 19:21:31.593130 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:31.593131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:31.593211 (+ 80us) server_negotiation.cc:407] Connection header received
0809 19:21:31.593258 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:31.593258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:31.593268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:31.593312 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:31.593579 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.594034 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.594659 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:31.594829 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:31.594869 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:31.597011 (+ 2142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:44417: BlockingRecv error: recv got EOF from 127.9.91.67:44417 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 19:21:32 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:32 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:32.371507 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:32.366999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:42841)
0809 19:21:32.367133 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:32.367145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:32.367204 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:32.367346 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:32.367347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:32.367523 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:32.367575 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:32.367577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.368237 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.368238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:32.368779 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:32.368781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.369121 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.369122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:32.369181 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:32.369694 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:32.369699 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:32.371432 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
W20260809 19:21:32.371596 23375 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:32.367082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:42841 (local address 127.9.91.126:40861)
0809 19:21:32.367195 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:32.367197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:32.367206 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:32.367264 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:32.367265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:32.367278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:32.367333 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:32.367649 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.368161 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.368858 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.369058 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.369119 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:32.371535 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:42841: BlockingRecv error: recv got EOF from 127.9.91.65:42841 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
Aug 09 19:21:32 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:32 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:32.484869 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:32.480623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:59147)
0809 19:21:32.480767 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:32.480779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:32.480830 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:32.480953 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:32.480954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:32.481101 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:32.481153 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:32.481155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.481736 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.481737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:32.482251 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:32.482252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.482520 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.482521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:32.482560 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:32.482997 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:32.483001 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:32.484814 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
W20260809 19:21:32.484958 23375 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:32.480746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:59147 (local address 127.9.91.126:40861)
0809 19:21:32.480804 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:32.480806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:32.480817 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:32.480882 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:32.480883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:32.480893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:32.480939 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:32.481203 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.481683 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.482302 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.482474 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.482525 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:32.484902 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:59147: BlockingRecv error: recv got EOF from 127.9.91.66:59147 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 19:21:32 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:32 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:32.601594 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:32.597389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:58991)
0809 19:21:32.597511 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:32.597521 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:32.597577 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:32.597690 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:32.597691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:32.597832 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:32.597878 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:32.597880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.598445 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.598446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:32.599013 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:32.599015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.599294 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.599295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:32.599331 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:32.599704 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:32.599708 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:32.601521 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260809 19:21:32.601660 23375 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:32.597461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:58991 (local address 127.9.91.126:40861)
0809 19:21:32.597531 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:32.597533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:32.597562 (+ 29us) server_negotiation.cc:407] Connection header received
0809 19:21:32.597626 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:32.597626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:32.597638 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:32.597679 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:32.597927 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.598394 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.599069 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:32.599245 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:32.599291 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:32.601608 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:58991: BlockingRecv error: recv got EOF from 127.9.91.67:58991 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 19:21:33 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:33 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:33.376431 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:33.371967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:36201)
0809 19:21:33.372106 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:33.372116 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:33.372168 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:33.372300 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:33.372300 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:33.372500 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:33.372556 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:33.372559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.373180 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.373181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:33.373711 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:33.373713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.374002 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.374003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:33.374043 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:33.374471 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:33.374477 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:33.376373 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
W20260809 19:21:33.376530 23383 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:33.372021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:36201 (local address 127.9.91.126:40861)
0809 19:21:33.372136 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:33.372138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:33.372151 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:21:33.372219 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:33.372219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:33.372230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:33.372285 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:33.372619 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.373121 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.373771 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.373949 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.373990 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:33.376463 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:36201: BlockingRecv error: recv got EOF from 127.9.91.65:36201 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
Aug 09 19:21:33 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:33 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:33.489821 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:33.485360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:53085)
0809 19:21:33.485552 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:33.485565 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:33.485642 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:33.485778 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:33.485779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:33.485964 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:33.486015 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:33.486017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.486719 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.486721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:33.487276 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:33.487277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.487599 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.487600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:33.487658 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:33.488114 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:33.488118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:33.489744 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
W20260809 19:21:33.489897 23383 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:33.485466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:53085 (local address 127.9.91.126:40861)
0809 19:21:33.485556 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:33.485558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:33.485635 (+ 77us) server_negotiation.cc:407] Connection header received
0809 19:21:33.485700 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:33.485701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:33.485714 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:33.485763 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:33.486089 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.486629 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.487347 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.487539 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.487595 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:33.489844 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:53085: BlockingRecv error: recv got EOF from 127.9.91.66:53085 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 19:21:33 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:33 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:33.606138 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:33.602026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:58829)
0809 19:21:33.602178 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:33.602188 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:33.602242 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:33.602355 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:33.602356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:33.602514 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:33.602565 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:33.602566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.603150 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.603151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:33.603676 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:33.603677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.603941 (+ 264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.603942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:33.603977 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:33.604421 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:33.604426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:33.606081 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
W20260809 19:21:33.606233 23383 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:33.602097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:58829 (local address 127.9.91.126:40861)
0809 19:21:33.602151 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:33.602153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:33.602238 (+ 85us) server_negotiation.cc:407] Connection header received
0809 19:21:33.602294 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:33.602294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:33.602305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:33.602343 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:33.602614 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.603103 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.603729 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:33.603897 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:33.603938 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:33.606168 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:58829: BlockingRecv error: recv got EOF from 127.9.91.67:58829 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 19:21:34 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:34 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:34.381143 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:34.376844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:46657)
0809 19:21:34.376967 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:34.376976 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:34.377022 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:34.377138 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:34.377139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:34.377299 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:34.377352 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:34.377354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.377966 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.377966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:34.378479 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:34.378480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.378761 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.378761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:34.378799 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:34.379326 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:34.379332 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:34.381084 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
W20260809 19:21:34.381238 23392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:34.376891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:46657 (local address 127.9.91.126:40861)
0809 19:21:34.377022 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:34.377023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:34.377052 (+ 29us) server_negotiation.cc:407] Connection header received
0809 19:21:34.377071 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:34.377072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:34.377082 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:34.377128 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:34.377402 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.377912 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.378529 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.378716 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.378757 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:34.381173 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:46657: BlockingRecv error: recv got EOF from 127.9.91.65:46657 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
Aug 09 19:21:34 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:34 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:34.494618 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:34.490281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:58709)
0809 19:21:34.490428 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:34.490439 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:34.490504 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:34.490639 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:34.490641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:34.490806 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:34.490854 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:34.490856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.491558 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.491559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:34.492089 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:34.492091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.492419 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.492420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:34.492478 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:34.492887 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:34.492891 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:34.494543 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
W20260809 19:21:34.494781 23392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:34.490360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:58709 (local address 127.9.91.126:40861)
0809 19:21:34.490426 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:34.490428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:34.490485 (+ 57us) server_negotiation.cc:407] Connection header received
0809 19:21:34.490555 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:34.490556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:34.490568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:34.490616 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:34.490947 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.491482 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.492172 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.492360 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.492415 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:34.494727 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:58709: BlockingRecv error: recv got EOF from 127.9.91.66:58709 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 19:21:34 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:34 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:34.610555 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:34.606535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:55223)
0809 19:21:34.606666 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:34.606683 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:34.606738 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:34.606850 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:34.606851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:34.607010 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:34.607058 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:34.607059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.607656 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.607657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:34.608169 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:34.608170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.608452 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.608452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:34.608489 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:34.608908 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:34.608913 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:34.610500 (+ 1587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
W20260809 19:21:34.610680 23392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:34.606594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:55223 (local address 127.9.91.126:40861)
0809 19:21:34.606641 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:34.606643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:34.606738 (+ 95us) server_negotiation.cc:407] Connection header received
0809 19:21:34.606778 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:34.606779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:34.606791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:34.606840 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:34.607109 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.607601 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.608223 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:34.608397 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:34.608449 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:34.610610 (+ 2161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:55223: BlockingRecv error: recv got EOF from 127.9.91.67:55223 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 19:21:35 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:35 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:35.386037 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:35.381584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:36673)
0809 19:21:35.381733 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:35.381747 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:35.381810 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:35.381953 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:35.381954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:35.382111 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:35.382162 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:35.382163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.382781 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.382782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:35.383372 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:35.383375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.383799 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.383801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:35.383872 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:35.384287 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:35.384291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:35.385980 (+ 1689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
W20260809 19:21:35.386168 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:35.381715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:36673 (local address 127.9.91.126:40861)
0809 19:21:35.381840 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:35.381842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:35.381849 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:35.381870 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:35.381871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:35.381882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:35.381944 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:35.382224 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.382726 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.383445 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.383712 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.383785 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:35.386084 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:36673: BlockingRecv error: recv got EOF from 127.9.91.65:36673 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
Aug 09 19:21:35 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:35 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:35.498988 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:35.495065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:60669)
0809 19:21:35.495183 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:35.495193 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:35.495247 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:35.495343 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:35.495344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:35.495476 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:35.495519 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:35.495520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.496210 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.496211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:35.496723 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:35.496724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.497001 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.497001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:35.497035 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:35.497400 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:35.497404 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:35.498889 (+ 1485us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
W20260809 19:21:35.499058 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:35.495128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:60669 (local address 127.9.91.126:40861)
0809 19:21:35.495173 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:35.495175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:35.495234 (+ 59us) server_negotiation.cc:407] Connection header received
0809 19:21:35.495295 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:35.495295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:35.495304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:35.495337 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:35.495565 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.496161 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.496781 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.496957 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.496996 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:35.499003 (+ 2007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:60669: BlockingRecv error: recv got EOF from 127.9.91.66:60669 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 19:21:35 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:35 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:35.614972 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:35.610970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:60085)
0809 19:21:35.611078 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:35.611088 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:35.611143 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:35.611238 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:35.611239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:35.611376 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:35.611422 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:35.611423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.611975 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.611976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:35.612499 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:35.612502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.612786 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.612787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:35.612824 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:35.613303 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:35.613307 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:35.614877 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
W20260809 19:21:35.615044 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:35.610981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:60085 (local address 127.9.91.126:40861)
0809 19:21:35.611035 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:35.611037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:35.611140 (+ 103us) server_negotiation.cc:407] Connection header received
0809 19:21:35.611181 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:35.611181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:35.611191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:35.611233 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:35.611465 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.611927 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.612561 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:35.612740 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:35.612783 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:35.614992 (+ 2209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:60085: BlockingRecv error: recv got EOF from 127.9.91.67:60085 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 19:21:36 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:36 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:36.390786 23404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:36.386439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:36383)
0809 19:21:36.386564 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:36.386574 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:36.386626 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:36.386750 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:36.386751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:36.386931 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:36.386992 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:36.386995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.387801 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.387802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:36.388334 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:36.388336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.388622 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.388623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:36.388659 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:36.389039 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:36.389044 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:36.390709 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
W20260809 19:21:36.390854 23405 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:36.386498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:36383 (local address 127.9.91.126:40861)
0809 19:21:36.386596 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:36.386598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:36.386609 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:36.386672 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:36.386673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:36.386683 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:36.386734 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:36.387060 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.387726 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.388390 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.388578 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.388621 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:36.390796 (+ 2175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:36383: BlockingRecv error: recv got EOF from 127.9.91.65:36383 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":45,"threads_started":1}
Aug 09 19:21:36 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:36 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:36.508127 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:36.499437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:33215)
0809 19:21:36.499558 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:36.499569 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:36.499634 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:36.499757 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:36.499758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:36.499915 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:36.499960 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:36.499962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.500637 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.500638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:36.501160 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:36.501162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.501528 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.501529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:36.501593 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:36.506477 (+ 4884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:36.506482 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:36.508055 (+ 1573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
W20260809 19:21:36.508213 23405 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:36.499514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:33215 (local address 127.9.91.126:40861)
0809 19:21:36.499564 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:36.499566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:36.499615 (+ 49us) server_negotiation.cc:407] Connection header received
0809 19:21:36.499680 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:36.499681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:36.499692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:36.499737 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:36.500027 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.500562 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.501238 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.501455 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.501506 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:36.508160 (+ 6654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:33215: BlockingRecv error: recv got EOF from 127.9.91.66:33215 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 19:21:36 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:36 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:36.619422 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:36.615366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:47459)
0809 19:21:36.615509 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:36.615529 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:36.615591 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:36.615700 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:36.615701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:36.615846 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:36.615892 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:36.615894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.616451 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.616451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:36.616972 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:36.616973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.617284 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.617285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:36.617323 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:36.617715 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:36.617719 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:36.619368 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
W20260809 19:21:36.619544 23405 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:36.615451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:47459 (local address 127.9.91.126:40861)
0809 19:21:36.615505 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:36.615507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:36.615583 (+ 76us) server_negotiation.cc:407] Connection header received
0809 19:21:36.615642 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:36.615643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:36.615652 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:36.615688 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:36.615938 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.616401 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.617026 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:36.617217 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:36.617262 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:36.619477 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:47459: BlockingRecv error: recv got EOF from 127.9.91.67:47459 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 19:21:37 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:37 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:37.395566 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:37.391267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:37259)
0809 19:21:37.391407 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:37.391420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:37.391475 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:37.391600 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:37.391601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:37.391776 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:37.391827 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:37.391829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.392415 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.392415 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:37.392942 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:37.392945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.393277 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.393279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:37.393344 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:37.393804 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:37.393810 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:37.395508 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
W20260809 19:21:37.395665 23412 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:37.391311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:37259 (local address 127.9.91.126:40861)
0809 19:21:37.391447 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:37.391448 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:37.391459 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:37.391523 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:37.391523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:37.391534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:37.391591 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:37.391877 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.392364 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.392994 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.393210 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.393267 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:37.395596 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:37259: BlockingRecv error: recv got EOF from 127.9.91.65:37259 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
Aug 09 19:21:37 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:37 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:37.513341 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:37.508561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:56761)
0809 19:21:37.508737 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:37.508750 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:37.508835 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:37.509003 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:37.509004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:37.509190 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:37.509255 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:37.509259 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.510098 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.510099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:37.510653 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:37.510656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.511059 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.511060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:37.511116 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:37.511522 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:37.511526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:37.513282 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
W20260809 19:21:37.513468 23412 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:37.508643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:56761 (local address 127.9.91.126:40861)
0809 19:21:37.508734 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:37.508736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:37.508816 (+ 80us) server_negotiation.cc:407] Connection header received
0809 19:21:37.508917 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:37.508918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:37.508930 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:37.508980 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:37.509335 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.510014 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.510740 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.510991 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.511059 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:37.513398 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:56761: BlockingRecv error: recv got EOF from 127.9.91.66:56761 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 19:21:37 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:37 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:37.624219 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:37.619889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:55199)
0809 19:21:37.620023 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:37.620035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:37.620091 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:37.620227 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:37.620228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:37.620410 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:37.620461 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:37.620463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.621102 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.621103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:37.621630 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:37.621631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.621944 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.621945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:37.621999 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:37.622421 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:37.622425 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:37.624146 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
W20260809 19:21:37.624377 23412 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:37.620051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:55199 (local address 127.9.91.126:40861)
0809 19:21:37.620119 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:37.620121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:37.620130 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:21:37.620153 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:37.620154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:37.620166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:37.620212 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:37.620532 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.621030 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.621706 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:37.621886 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:37.621940 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:37.624312 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:55199: BlockingRecv error: recv got EOF from 127.9.91.67:55199 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 19:21:38 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:38 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:38.400421 23415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:38.395975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:43003)
0809 19:21:38.396106 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:38.396116 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:38.396167 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:38.396282 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:38.396282 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:38.396448 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:38.396497 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:38.396499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.397285 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.397286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:38.397809 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:38.397811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.398105 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.398106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:38.398143 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:38.398574 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:38.398581 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:38.400353 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
W20260809 19:21:38.400514 23416 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:38.396043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:43003 (local address 127.9.91.126:40861)
0809 19:21:38.396142 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:38.396143 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:38.396154 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:38.396207 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:38.396208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:38.396218 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:38.396272 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:38.396546 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.397221 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.397867 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.398054 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.398094 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:38.400439 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:43003: BlockingRecv error: recv got EOF from 127.9.91.65:43003 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":51,"threads_started":1}
Aug 09 19:21:38 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:38 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:38.518369 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:38.513743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:53217)
0809 19:21:38.513929 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:38.513938 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:38.513997 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:38.514123 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:38.514124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:38.514298 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:38.514341 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:38.514344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.515064 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.515065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:38.515627 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:38.515630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.515969 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.515970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:38.516029 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:38.516449 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:38.516455 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:38.518267 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
W20260809 19:21:38.518395 23416 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:38.513795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:53217 (local address 127.9.91.126:40861)
0809 19:21:38.513895 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:38.513898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:38.513994 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:21:38.514048 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:38.514049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:38.514062 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:38.514116 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:38.514414 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.514990 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.515702 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.515910 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.515964 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:38.518349 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:53217: BlockingRecv error: recv got EOF from 127.9.91.66:53217 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 19:21:38 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:38 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:38.628868 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:38.624640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:50849)
0809 19:21:38.624761 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:38.624773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:38.624845 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:38.624974 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:38.624975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:38.625130 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:38.625173 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:38.625175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.625780 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.625781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:38.626307 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:38.626309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.626625 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.626626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:38.626683 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:38.627155 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:38.627159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:38.628798 (+ 1639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
W20260809 19:21:38.629045 23416 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:38.624723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:50849 (local address 127.9.91.126:40861)
0809 19:21:38.624794 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:38.624797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:38.624827 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:21:38.624900 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:38.624901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:38.624913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:38.624959 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:38.625243 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.625710 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.626384 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:38.626564 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:38.626620 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:38.628983 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:50849: BlockingRecv error: recv got EOF from 127.9.91.67:50849 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 19:21:39 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:39 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:39.405202 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:39.400903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:59355)
0809 19:21:39.401049 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:39.401060 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:39.401127 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:39.401334 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:39.401335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:39.401498 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:39.401548 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:39.401550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.402200 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.402200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:39.402730 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:39.402732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.403050 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.403051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:39.403088 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:39.403511 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:39.403515 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:39.405132 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260809 19:21:39.405308 23423 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:39.400949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:59355 (local address 127.9.91.126:40861)
0809 19:21:39.401061 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:39.401063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:39.401109 (+ 46us) server_negotiation.cc:407] Connection header received
0809 19:21:39.401195 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:39.401197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:39.401211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:39.401272 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:39.401611 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.402135 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.402790 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.402996 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.403049 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:39.405235 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:59355: BlockingRecv error: recv got EOF from 127.9.91.65:59355 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
Aug 09 19:21:39 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:39 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:39.522876 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:39.518777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:45643)
0809 19:21:39.518935 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:39.518945 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:39.518999 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:39.519108 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:39.519109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:39.519275 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:39.519323 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:39.519325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.519922 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.519922 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:39.520442 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:39.520443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.520712 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.520712 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:39.520749 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:39.521162 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:39.521166 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:39.522804 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
W20260809 19:21:39.522990 23423 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:39.518825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:45643 (local address 127.9.91.126:40861)
0809 19:21:39.518886 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:39.518888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:39.518999 (+ 111us) server_negotiation.cc:407] Connection header received
0809 19:21:39.519040 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:39.519040 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:39.519050 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:39.519097 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:39.519373 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.519870 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.520493 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.520666 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.520709 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:39.522904 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:45643: BlockingRecv error: recv got EOF from 127.9.91.66:45643 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 19:21:39 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:39 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:39.633522 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:39.629296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:48905)
0809 19:21:39.629424 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:39.629434 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:39.629491 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:39.629596 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:39.629597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:39.629764 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:39.629818 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:39.629840 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.630433 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.630434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:39.631016 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:39.631018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.631309 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.631309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:39.631347 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:39.631773 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:39.631778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:39.633431 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
W20260809 19:21:39.633561 23423 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:39.629354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:48905 (local address 127.9.91.126:40861)
0809 19:21:39.629427 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:39.629429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:39.629486 (+ 57us) server_negotiation.cc:407] Connection header received
0809 19:21:39.629530 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:39.629530 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:39.629541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:39.629590 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:39.629898 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.630371 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.631082 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:39.631260 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:39.631305 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:39.633508 (+ 2203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:48905: BlockingRecv error: recv got EOF from 127.9.91.67:48905 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 19:21:40 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:40 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:40.409812 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:40.405621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:57577)
0809 19:21:40.405749 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:40.405758 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:40.405810 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:40.405923 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:40.405924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:40.406091 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:40.406140 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:40.406142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.406737 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.406738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:40.407298 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:40.407299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.407585 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.407586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:40.407623 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:40.408003 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:40.408007 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:40.409724 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
W20260809 19:21:40.409888 23428 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:40.405698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:57577 (local address 127.9.91.126:40861)
0809 19:21:40.405812 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:40.405814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:40.405825 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:40.405847 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:40.405847 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:40.405858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:40.405914 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:40.406197 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.406686 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.407350 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.407533 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.407575 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:40.409826 (+ 2251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:57577: BlockingRecv error: recv got EOF from 127.9.91.65:57577 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":50,"threads_started":1}
Aug 09 19:21:40 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:40 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:40.527498 23429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:40.523293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:36801)
0809 19:21:40.523409 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:40.523420 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:40.523474 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:40.523592 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:40.523593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:40.523759 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:40.523804 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:40.523806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.524427 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.524428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:40.524977 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:40.524979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.525308 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.525310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:40.525368 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:40.525778 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:40.525782 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:40.527413 (+ 1631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
W20260809 19:21:40.527562 23428 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:40.523405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:36801 (local address 127.9.91.126:40861)
0809 19:21:40.523450 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:40.523452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:40.523460 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:40.523523 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:40.523523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:40.523535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:40.523582 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:40.523872 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.524354 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.525050 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.525249 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.525302 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:40.527510 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:36801: BlockingRecv error: recv got EOF from 127.9.91.66:36801 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 19:21:40 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:40 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:40.637766 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:40.633910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:39193)
0809 19:21:40.634024 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:40.634033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:40.634089 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:40.634188 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:40.634188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:40.634318 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:40.634359 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:40.634361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.634897 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.634898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:40.635434 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:40.635435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.635692 (+ 257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.635693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:40.635728 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:40.636120 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:40.636124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:40.637704 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
W20260809 19:21:40.637833 23428 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:40.633969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:39193 (local address 127.9.91.126:40861)
0809 19:21:40.634026 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:40.634028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:40.634087 (+ 59us) server_negotiation.cc:407] Connection header received
0809 19:21:40.634126 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:40.634126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:40.634135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:40.634178 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:40.634405 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.634849 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.635485 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:40.635646 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:40.635684 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:40.637779 (+ 2095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:39193: BlockingRecv error: recv got EOF from 127.9.91.67:39193 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 19:21:41 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:41 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:41.415444 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:41.410243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:42863)
0809 19:21:41.410349 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:41.410358 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:41.410406 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:41.410557 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:41.410557 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:41.410754 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:41.410802 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:41.410804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.411600 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.411601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:41.412388 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:41.412392 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.412873 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.412874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:41.412937 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:41.413360 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:41.413366 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:41.415365 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":53,"threads_started":1}
W20260809 19:21:41.415531 23432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:41.410294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:42863 (local address 127.9.91.126:40861)
0809 19:21:41.410410 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:41.410412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:41.410423 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:21:41.410450 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:41.410451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:41.410464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:41.410556 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:41.410877 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.411508 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.412484 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.412788 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.412863 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:41.415467 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:42863: BlockingRecv error: recv got EOF from 127.9.91.65:42863 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
Aug 09 19:21:41 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:41 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:41.534862 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:41.527974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:45761)
0809 19:21:41.528114 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:41.528124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:41.528193 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:41.528304 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:41.528305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:41.528478 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:41.528526 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:41.528528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.529114 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.529114 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:41.529669 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:41.529672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.529966 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.529967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:41.530004 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:41.533213 (+ 3209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:41.533218 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:41.534804 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
W20260809 19:21:41.534974 23432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:41.528041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:45761 (local address 127.9.91.126:40861)
0809 19:21:41.528113 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:41.528115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:41.528170 (+ 55us) server_negotiation.cc:407] Connection header received
0809 19:21:41.528236 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:41.528236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:41.528246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:41.528292 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:41.528577 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.529052 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.529728 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.529914 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.529964 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:41.534893 (+ 4929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:45761: BlockingRecv error: recv got EOF from 127.9.91.66:45761 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 19:21:41 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:41 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.67@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:41.642267 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:41.638172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.67:33083)
0809 19:21:41.638313 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:41.638324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:41.638382 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:41.638496 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:41.638497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:41.638661 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:41.638711 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:41.638712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.639366 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.639367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:41.639907 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:41.639909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.640183 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.640184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:41.640221 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:41.640637 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:41.640640 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:41.642209 (+ 1569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260809 19:21:41.642354 23432 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:41.638215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:33083 (local address 127.9.91.126:40861)
0809 19:21:41.638257 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:41.638260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:41.638378 (+ 118us) server_negotiation.cc:407] Connection header received
0809 19:21:41.638421 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:41.638421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:41.638432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:41.638476 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:41.638759 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.639297 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.639961 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:41.640131 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:41.640172 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:41.642290 (+ 2118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.67:33083: BlockingRecv error: recv got EOF from 127.9.91.67:33083 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 19:21:42 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:42 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.65@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:42.420928 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:42.416173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.65:40035)
0809 19:21:42.416303 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:42.416313 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:42.416360 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:42.416534 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:42.416536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:42.416751 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:42.416807 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:42.416809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.417581 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.417582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:42.418169 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:42.418173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.418512 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.418513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:42.418552 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:42.419068 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:42.419073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:42.420850 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
W20260809 19:21:42.421010 23437 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:42.416228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:40035 (local address 127.9.91.126:40861)
0809 19:21:42.416342 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:42.416344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:42.416352 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:21:42.416404 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:42.416405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:42.416416 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:42.416478 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:42.416882 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.417476 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.418244 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.418441 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.418491 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:42.420949 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.65:40035: BlockingRecv error: recv got EOF from 127.9.91.65:40035 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
Aug 09 19:21:42 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 19:21:42 dist-test-slave-1vmg krb5kdc[21797](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.91.66@KRBTEST.COM for oryx/127.9.91.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 19:21:42.539755 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:42.535343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:40861 (local address 127.9.91.66:44797)
0809 19:21:42.535503 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:42.535514 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:42.535576 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:42.535734 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:42.535735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:42.535909 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:42.535964 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:42.535966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.536564 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.536564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:42.537106 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:42.537108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.537385 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.537386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:42.537423 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:42.537896 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:42.537902 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:42.539666 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.91.126:40861: Server oryx/127.9.91.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
W20260809 19:21:42.539839 23437 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:42.535402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:44797 (local address 127.9.91.126:40861)
0809 19:21:42.535449 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:42.535450 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:42.535571 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:21:42.535635 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:42.535635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:42.535645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:42.535691 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:42.536031 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.536509 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.537165 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.537336 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.537377 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:42.539784 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.91.66:44797: BlockingRecv error: recv got EOF from 127.9.91.66:44797 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:21:42.564730 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 22725
I20260809 19:21:42.575706 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 22952
I20260809 19:21:42.585039 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 23147
I20260809 19:21:42.592386 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 22611
2026-08-09T19:21:42Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17455 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T19:21:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:42Z Disabled control of system clock
I20260809 19:21:42.622043 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36989
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 19:21:42.700291 23447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:42.700464 23447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:42.700482 23447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:42.701602 23447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:42.701642 23447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:42.701653 23447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:42.701678 23447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:42.701699 23447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:42.702996 23447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36989
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:42.703200 23447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:42.703449 23447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:42.707911 23454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:42.707832 23457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:42.708045 23447 server_base.cc:1034] running on GCE node
W20260809 19:21:42.707844 23455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:42.708392 23447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:42.708649 23447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:42.709769 23447 hybrid_clock.cc:648] HybridClock initialized: now 1786303302709762 us; error 28 us; skew 500 ppm
I20260809 19:21:42.710836 23447 webserver.cc:460] Webserver started at http://127.9.91.126:35393/ using document root <none> and password file <none>
I20260809 19:21:42.711046 23447 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:42.711093 23447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:42.711208 23447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:42.712141 23447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "d3dbf1e1365d432fa77735c53e86d0ae"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "ebeff312399271b26c23a2d21bff42cd"
server_key_iv: "914d5ade60bfcda9903aec4b81891f3f"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.712443 23447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "d3dbf1e1365d432fa77735c53e86d0ae"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "ebeff312399271b26c23a2d21bff42cd"
server_key_iv: "914d5ade60bfcda9903aec4b81891f3f"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.713681 23447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:42.714323 23468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:42.714445 23447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:42.714520 23447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "d3dbf1e1365d432fa77735c53e86d0ae"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "ebeff312399271b26c23a2d21bff42cd"
server_key_iv: "914d5ade60bfcda9903aec4b81891f3f"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.714589 23447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:42.746461 23447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:42.746702 23447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:42.746821 23447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:42.750415 23447 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:36989
I20260809 19:21:42.750449 23548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:36989 every 8 connection(s)
I20260809 19:21:42.750725 23447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:42.751332 23551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:42.753774 23551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae: Bootstrap starting.
I20260809 19:21:42.754321 23551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:42.754587 23551 log.cc:824] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:42.755218 23551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae: No bootstrap required, opened a new log
I20260809 19:21:42.756119 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 23447
I20260809 19:21:42.756199 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:42.756363 9581 external_mini_cluster.cc:1428] Setting key c1c5d93813b85b98460988f831d568e7
I20260809 19:21:42.756913 23551 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36989 } }
I20260809 19:21:42.757072 23551 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:42.757115 23551 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3dbf1e1365d432fa77735c53e86d0ae, State: Initialized, Role: FOLLOWER
I20260809 19:21:42.757244 23551 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36989 } }
I20260809 19:21:42.757320 23551 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:42.757359 23551 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:42.757408 23551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:42.758010 23551 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36989 } }
I20260809 19:21:42.758100 23551 leader_election.cc:302] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3dbf1e1365d432fa77735c53e86d0ae; no voters:
I20260809 19:21:42.758257 23551 leader_election.cc:288] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:42.758301 23560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:42.758441 23560 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [term 1 LEADER]: Becoming Leader. State: Replica: d3dbf1e1365d432fa77735c53e86d0ae, State: Running, Role: LEADER
I20260809 19:21:42.758512 23551 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:42.758562 23560 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36989 } }
I20260809 19:21:42.759033 23560 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3dbf1e1365d432fa77735c53e86d0ae. Latest consensus state: current_term: 1 leader_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36989 } } }
I20260809 19:21:42.759043 23561 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3dbf1e1365d432fa77735c53e86d0ae" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36989 } } }
I20260809 19:21:42.759205 23560 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:42.759217 23561 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:42.759586 23576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:42.760326 23576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:42.761948 23576 catalog_manager.cc:1347] Generated new cluster ID: f6c9976819a642a7a5ed76dc001cce0f
I20260809 19:21:42.762001 23576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:42.762540 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:42.756851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56194 (local address 127.9.91.126:36989)
0809 19:21:42.757078 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:42.757087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:42.757131 (+ 44us) server_negotiation.cc:407] Connection header received
0809 19:21:42.757236 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:42.757243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:42.757310 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:42.757453 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:42.757803 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.759391 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.760467 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.761049 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.761161 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:42.761829 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:42.761833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:42.761833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:42.762029 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:42.762070 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:42.762085 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:42.762158 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:42.762250 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 19:21:42.762330 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260809 19:21:42.777732 23576 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:42.778192 23576 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:42.781155 23576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3dbf1e1365d432fa77735c53e86d0ae: Generated new TSK 0
I20260809 19:21:42.781296 23576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:42.789140 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36989
--builtin_ntp_servers=127.9.91.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 19:21:42.868273 23590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:42.868458 23590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:42.868484 23590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:42.869562 23590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:42.869599 23590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:42.869630 23590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:42.870886 23590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:42.871145 23590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:42.871407 23590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:42.876830 23595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:42.876856 23597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:42.877116 23599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:42.877141 23590 server_base.cc:1034] running on GCE node
I20260809 19:21:42.877354 23590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:42.877556 23590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:42.878692 23590 hybrid_clock.cc:648] HybridClock initialized: now 1786303302878681 us; error 30 us; skew 500 ppm
I20260809 19:21:42.879796 23590 webserver.cc:460] Webserver started at http://127.9.91.65:37265/ using document root <none> and password file <none>
I20260809 19:21:42.879992 23590 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:42.880039 23590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:42.880177 23590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:42.881137 23590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "9f6eea972dcc4b11b40d2a599aefc700"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "e9f1c75b11e812e728ba80a93b5fbf3e"
server_key_iv: "893d5ddf2b40ed08ee5c83191edf069d"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.881453 23590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "9f6eea972dcc4b11b40d2a599aefc700"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "e9f1c75b11e812e728ba80a93b5fbf3e"
server_key_iv: "893d5ddf2b40ed08ee5c83191edf069d"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.882673 23590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:42.883417 23607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:42.883559 23590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:42.883642 23590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "9f6eea972dcc4b11b40d2a599aefc700"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "e9f1c75b11e812e728ba80a93b5fbf3e"
server_key_iv: "893d5ddf2b40ed08ee5c83191edf069d"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.883708 23590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:42.894528 23590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:42.894753 23590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:42.894870 23590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:42.895088 23590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:42.895373 23590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:42.895406 23590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:42.895431 23590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:42.895448 23590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:42.900827 23590 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:38907
I20260809 19:21:42.900862 23792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:38907 every 8 connection(s)
I20260809 19:21:42.901146 23590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:42.904376 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 23590
I20260809 19:21:42.904479 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:42.904676 9581 external_mini_cluster.cc:1428] Setting key c3dbed713bc238cd0290aa8311759514
I20260809 19:21:42.905536 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36989
--builtin_ntp_servers=127.9.91.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 19:21:42.905887 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:42.901746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35931 (local address 127.9.91.126:36989)
0809 19:21:42.901799 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:42.901801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:42.902017 (+ 216us) server_negotiation.cc:407] Connection header received
0809 19:21:42.902430 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:42.902431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:42.902440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:42.902490 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:42.903123 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.903600 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.904629 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.904820 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.904888 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:42.905636 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:42.905637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:42.905637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:42.905642 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:42.905657 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:42.905660 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:42.905704 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:42.905776 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 19:21:42.905809 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:21:42.906061 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:42.901736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36989 (local address 127.9.91.65:35931)
0809 19:21:42.901907 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:42.901943 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:42.902360 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:42.902536 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:42.902538 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:42.902546 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:42.903047 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:42.903052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.903658 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.903659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:42.904544 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:42.904547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:42.904901 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:42.904903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:42.905043 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:42.905509 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:42.905510 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:42.905536 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:42.905556 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:42.905711 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:42.905716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:42.905719 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:42.905779 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:21:42.905877 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260809 19:21:42.907493 23793 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36989
I20260809 19:21:42.907593 23793 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:42.907773 23793 heartbeater.cc:511] Master 127.9.91.126:36989 requested a full tablet report, sending...
I20260809 19:21:42.908253 23494 ts_manager.cc:194] Registered new tserver with Master: 9f6eea972dcc4b11b40d2a599aefc700 (127.9.91.65:38907)
I20260809 19:21:42.908818 23494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.65:35931
W20260809 19:21:42.984792 23798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:42.984969 23798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:42.984997 23798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:42.986099 23798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:42.986135 23798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:42.986166 23798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:42.987486 23798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:42.987696 23798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:42.987943 23798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:42.992424 23806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:42.992429 23805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:42.992441 23808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:42.992713 23798 server_base.cc:1034] running on GCE node
I20260809 19:21:42.992903 23798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:42.993086 23798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:42.994206 23798 hybrid_clock.cc:648] HybridClock initialized: now 1786303302994201 us; error 30 us; skew 500 ppm
I20260809 19:21:42.995359 23798 webserver.cc:460] Webserver started at http://127.9.91.66:42799/ using document root <none> and password file <none>
I20260809 19:21:42.995544 23798 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:42.995589 23798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:42.995702 23798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:42.996634 23798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "9baf303661714fce89995f9ee5177e3c"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "50f72ab62e7ec94b86110aef6dcdc800"
server_key_iv: "b1684300aa6027c1717364a593c01473"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.996990 23798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "9baf303661714fce89995f9ee5177e3c"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "50f72ab62e7ec94b86110aef6dcdc800"
server_key_iv: "b1684300aa6027c1717364a593c01473"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.998150 23798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:21:42.998740 23814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:42.998898 23798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:42.999017 23798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "9baf303661714fce89995f9ee5177e3c"
format_stamp: "Formatted at 2026-08-09 19:21:42 on dist-test-slave-1vmg"
server_key: "50f72ab62e7ec94b86110aef6dcdc800"
server_key_iv: "b1684300aa6027c1717364a593c01473"
server_key_version: "encryptionkey@0"
I20260809 19:21:42.999075 23798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:43.021335 23798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:43.021579 23798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:43.021682 23798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:43.021867 23798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:43.022138 23798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:43.022171 23798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.022199 23798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:43.022219 23798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.028074 23798 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:36451
I20260809 19:21:43.028141 23990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:36451 every 8 connection(s)
I20260809 19:21:43.028398 23798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:43.031041 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 23798
I20260809 19:21:43.031147 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:43.031338 9581 external_mini_cluster.cc:1428] Setting key 7add009c0454e361ac3b20c547e7e22a
I20260809 19:21:43.032266 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36989
--builtin_ntp_servers=127.9.91.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 19:21:43.033550 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.029057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:36001 (local address 127.9.91.126:36989)
0809 19:21:43.029146 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.029149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.029342 (+ 193us) server_negotiation.cc:407] Connection header received
0809 19:21:43.029775 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.029776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.029788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.029839 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.030412 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.030939 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.032229 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.032448 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.032515 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:43.033310 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:43.033312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:43.033312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:43.033316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:43.033331 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:43.033335 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:43.033382 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:43.033462 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 19:21:43.033487 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 19:21:43.033759 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.029034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36989 (local address 127.9.91.66:36001)
0809 19:21:43.029234 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:43.029265 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:43.029699 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:43.029895 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:43.029898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:43.029906 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:43.030353 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:43.030357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.031063 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.031064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:43.032143 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:43.032146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.032543 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.032544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:43.032686 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:43.033163 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:43.033164 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:43.033197 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:43.033221 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:43.033391 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:43.033395 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:43.033397 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:43.033459 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:21:43.033558 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
I20260809 19:21:43.034248 23992 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36989
I20260809 19:21:43.034341 23992 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:43.034528 23992 heartbeater.cc:511] Master 127.9.91.126:36989 requested a full tablet report, sending...
I20260809 19:21:43.034868 23494 ts_manager.cc:194] Registered new tserver with Master: 9baf303661714fce89995f9ee5177e3c (127.9.91.66:36451)
I20260809 19:21:43.035315 23494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.66:36001
W20260809 19:21:43.114382 23997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:43.114573 23997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:43.114594 23997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:43.115727 23997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:43.115772 23997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:43.115797 23997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:43.117084 23997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:43.117259 23997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:43.117492 23997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:21:43.122048 23997 server_base.cc:1034] running on GCE node
W20260809 19:21:43.122038 24012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.122016 24007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.121992 24009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:43.122437 23997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:43.122669 23997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:43.123799 23997 hybrid_clock.cc:648] HybridClock initialized: now 1786303303123782 us; error 33 us; skew 500 ppm
I20260809 19:21:43.124925 23997 webserver.cc:460] Webserver started at http://127.9.91.67:43801/ using document root <none> and password file <none>
I20260809 19:21:43.125128 23997 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:43.125174 23997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:43.125290 23997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:43.126235 23997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "272089cb962f4c1d9dd9d3d84de75c2d"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "94aaf591dbcbb1d04c8ab695a01f2f74"
server_key_iv: "60be6aa08512a8b74c4e6260ca6d9791"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.126539 23997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "272089cb962f4c1d9dd9d3d84de75c2d"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "94aaf591dbcbb1d04c8ab695a01f2f74"
server_key_iv: "60be6aa08512a8b74c4e6260ca6d9791"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.127722 23997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:43.128334 24020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.128475 23997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:43.128552 23997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "272089cb962f4c1d9dd9d3d84de75c2d"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "94aaf591dbcbb1d04c8ab695a01f2f74"
server_key_iv: "60be6aa08512a8b74c4e6260ca6d9791"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.128613 23997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:43.149670 23997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:43.149885 23997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:43.149999 23997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:43.150182 23997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:43.150458 23997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:43.150497 23997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.150523 23997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:43.150542 23997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.155825 23997 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:38333
I20260809 19:21:43.155886 24196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:38333 every 8 connection(s)
I20260809 19:21:43.156129 23997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:43.157950 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 23997
I20260809 19:21:43.158032 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:43.158200 9581 external_mini_cluster.cc:1428] Setting key be80dfbbf1e19bfa66a09cbf8a35055e
I20260809 19:21:43.161042 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.156736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:34661 (local address 127.9.91.126:36989)
0809 19:21:43.156797 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.156799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.156976 (+ 177us) server_negotiation.cc:407] Connection header received
0809 19:21:43.157433 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.157434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.157447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.157499 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.158144 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.158876 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.159926 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.160135 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.160186 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:43.160878 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:43.160879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:43.160879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:43.160882 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:43.160892 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:43.160895 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:43.160924 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:43.160968 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 19:21:43.160988 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:21:43.161237 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.156693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36989 (local address 127.9.91.67:34661)
0809 19:21:43.156878 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:43.156907 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:43.157342 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:43.157561 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:43.157565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:43.157575 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:43.158049 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:43.158056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.158998 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.158999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:43.159854 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:43.159857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:43.160209 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:43.160210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:43.160313 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:43.160768 (+ 455us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:43.160769 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:43.160795 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:43.160815 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:43.160925 (+ 110us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:43.160927 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:43.160929 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:43.160969 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 19:21:43.161038 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260809 19:21:43.161656 24197 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36989
I20260809 19:21:43.161739 24197 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:43.161909 24197 heartbeater.cc:511] Master 127.9.91.126:36989 requested a full tablet report, sending...
I20260809 19:21:43.162298 23494 ts_manager.cc:194] Registered new tserver with Master: 272089cb962f4c1d9dd9d3d84de75c2d (127.9.91.67:38333)
I20260809 19:21:43.162755 23494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.67:34661
I20260809 19:21:43.169803 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:21:43.171638 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 23590
W20260809 19:21:43.171659 23559 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:43.171093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.9.91.126:36989)
0809 19:21:43.171146 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.171148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.171257 (+ 109us) server_negotiation.cc:407] Connection header received
0809 19:21:43.171326 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.171326 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.171339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.171396 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.171598 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56208: BlockingRecv error: recv got EOF from 127.0.0.1:56208 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:21:43.180616 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 23798
I20260809 19:21:43.192734 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 23997
I20260809 19:21:43.200464 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 23447
2026-08-09T19:21:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (607 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T19:21:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:43Z Disabled control of system clock
I20260809 19:21:43.229557 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46125
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:35973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:46125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:21:43.308331 24212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:43.308509 24212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:43.308528 24212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:43.309615 24212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:43.309656 24212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:43.309669 24212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:43.309692 24212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:43.309716 24212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:43.311039 24212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:46125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:43.311249 24212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:43.311496 24212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:43.315815 24220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.315851 24226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.315831 24224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:43.316058 24212 server_base.cc:1034] running on GCE node
I20260809 19:21:43.316288 24212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:43.316550 24212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:43.317691 24212 hybrid_clock.cc:648] HybridClock initialized: now 1786303303317675 us; error 25 us; skew 500 ppm
I20260809 19:21:43.319057 24212 webserver.cc:460] Webserver started at http://127.9.91.126:36287/ using document root <none> and password file <none>
I20260809 19:21:43.319298 24212 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:43.319356 24212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:43.319482 24212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:43.320632 24212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "16f02f5836384089bc2b2501d5718211"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "2c39456966e687b67c0ff1d636487b0a"
server_key_iv: "36254623114d78f5fd22a288c27f4834"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.320955 24212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "16f02f5836384089bc2b2501d5718211"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "2c39456966e687b67c0ff1d636487b0a"
server_key_iv: "36254623114d78f5fd22a288c27f4834"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.322149 24212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:43.322788 24233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.322973 24212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:43.323061 24212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "16f02f5836384089bc2b2501d5718211"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "2c39456966e687b67c0ff1d636487b0a"
server_key_iv: "36254623114d78f5fd22a288c27f4834"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.323127 24212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:43.326478 24212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:43.326715 24212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:43.326834 24212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:43.330940 24212 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:46125
I20260809 19:21:43.330976 24318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:46125 every 8 connection(s)
I20260809 19:21:43.331331 24212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:43.332044 24319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:43.333189 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 24212
I20260809 19:21:43.333284 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:43.333467 9581 external_mini_cluster.cc:1428] Setting key 06136f434cccad9c5625dbfc1c625120
I20260809 19:21:43.336000 24319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211: Bootstrap starting.
I20260809 19:21:43.336031 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.334205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.9.91.126:46125)
0809 19:21:43.334388 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.334397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.334427 (+ 30us) server_negotiation.cc:407] Connection header received
0809 19:21:43.334876 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.334885 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.334960 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.335098 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.335636 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:43.335640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:43.335640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:43.335644 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:43.335694 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:43.335707 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:43.335782 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:43.335811 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 19:21:43.335863 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260809 19:21:43.336699 24319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:43.336933 24319 log.cc:824] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:43.337487 24319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211: No bootstrap required, opened a new log
I20260809 19:21:43.338712 24319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f02f5836384089bc2b2501d5718211" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46125 } }
I20260809 19:21:43.338827 24319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:43.338855 24319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f02f5836384089bc2b2501d5718211, State: Initialized, Role: FOLLOWER
I20260809 19:21:43.338974 24319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f02f5836384089bc2b2501d5718211" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46125 } }
I20260809 19:21:43.339059 24319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:43.339082 24319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:43.339113 24319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:43.339550 24319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f02f5836384089bc2b2501d5718211" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46125 } }
I20260809 19:21:43.339632 24319 leader_election.cc:302] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16f02f5836384089bc2b2501d5718211; no voters:
I20260809 19:21:43.339758 24319 leader_election.cc:288] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:43.339984 24319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:43.340026 24327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:43.340147 24327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [term 1 LEADER]: Becoming Leader. State: Replica: 16f02f5836384089bc2b2501d5718211, State: Running, Role: LEADER
I20260809 19:21:43.340440 24327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f02f5836384089bc2b2501d5718211" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46125 } }
I20260809 19:21:43.341395 24327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16f02f5836384089bc2b2501d5718211. Latest consensus state: current_term: 1 leader_uuid: "16f02f5836384089bc2b2501d5718211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f02f5836384089bc2b2501d5718211" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46125 } } }
I20260809 19:21:43.341565 24327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:43.341785 24346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:43.341871 24328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16f02f5836384089bc2b2501d5718211" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f02f5836384089bc2b2501d5718211" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46125 } } }
I20260809 19:21:43.341943 24328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:43.342226 24346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:43.344449 24353 mvcc.cc:205] Tried to move back new op lower bound from 7316698330493272064 to 7316698330484686848. Current Snapshot: MvccSnapshot[applied={T|T < 7316698330493272064}]
I20260809 19:21:43.344937 24346 catalog_manager.cc:1347] Generated new cluster ID: 3a33209846a6468eabd6f83820d70251
I20260809 19:21:43.345053 24346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:21:43.363245 24346 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:43.363387 24346 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:43.369403 24346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16f02f5836384089bc2b2501d5718211: Generated new TSK 0
I20260809 19:21:43.369558 24346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:21:43.376415 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46125
--builtin_ntp_servers=127.9.91.84:35973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:21:43.456478 24357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:43.456672 24357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:43.456702 24357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:43.457787 24357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:43.457823 24357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:43.457855 24357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:43.459163 24357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:43.459369 24357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:43.459625 24357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:43.465113 24371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.465171 24366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.465181 24367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:43.465513 24357 server_base.cc:1034] running on GCE node
I20260809 19:21:43.465700 24357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:43.465925 24357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:43.467072 24357 hybrid_clock.cc:648] HybridClock initialized: now 1786303303467051 us; error 35 us; skew 500 ppm
I20260809 19:21:43.468397 24357 webserver.cc:460] Webserver started at http://127.9.91.65:33875/ using document root <none> and password file <none>
I20260809 19:21:43.468652 24357 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:43.468708 24357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:43.468829 24357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:43.470153 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "25bb0825a2994de9ad07ce92dbd24704"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "919a456c9ba13f0bb2632c8c56e5a52e"
server_key_iv: "cb0d98c835bf8b149521cb14aebbc751"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.470582 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "25bb0825a2994de9ad07ce92dbd24704"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "919a456c9ba13f0bb2632c8c56e5a52e"
server_key_iv: "cb0d98c835bf8b149521cb14aebbc751"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.471787 24357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:21:43.472394 24379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.472541 24357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:21:43.472616 24357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "25bb0825a2994de9ad07ce92dbd24704"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "919a456c9ba13f0bb2632c8c56e5a52e"
server_key_iv: "cb0d98c835bf8b149521cb14aebbc751"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.472684 24357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:43.475807 24357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:43.476063 24357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:43.476177 24357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:43.476372 24357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:43.476640 24357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:43.476672 24357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.476699 24357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:43.476723 24357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.482153 24357 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:45829
I20260809 19:21:43.482210 24565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:45829 every 8 connection(s)
I20260809 19:21:43.482466 24357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:21:43.485109 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.483080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:60767 (local address 127.9.91.126:46125)
0809 19:21:43.483120 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.483123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.483336 (+ 213us) server_negotiation.cc:407] Connection header received
0809 19:21:43.483776 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.483777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.483785 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.483835 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.484877 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:43.484879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:43.484879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:43.484883 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:43.484900 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:43.484904 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:43.484953 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:43.485028 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 19:21:43.485041 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:21:43.485282 24571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.483076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46125 (local address 127.9.91.65:60767)
0809 19:21:43.483237 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:43.483266 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:43.483677 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:43.483896 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:43.483900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:43.483910 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:43.484742 (+ 832us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:43.484742 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:43.484768 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:43.484792 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:43.484957 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:43.484962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:43.484965 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:43.485042 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:21:43.485100 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260809 19:21:43.485704 24567 heartbeater.cc:347] Connected to a master server at 127.9.91.126:46125
I20260809 19:21:43.485777 24567 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:43.485910 24567 heartbeater.cc:511] Master 127.9.91.126:46125 requested a full tablet report, sending...
I20260809 19:21:43.486335 24260 ts_manager.cc:194] Registered new tserver with Master: 25bb0825a2994de9ad07ce92dbd24704 (127.9.91.65:45829)
I20260809 19:21:43.491475 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 24357
I20260809 19:21:43.491564 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:43.491731 9581 external_mini_cluster.cc:1428] Setting key bbb06f46b18b1521984906a67ccf8f04
I20260809 19:21:43.492514 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46125
--builtin_ntp_servers=127.9.91.84:35973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:21:43.571081 24572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:43.571249 24572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:43.571270 24572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:43.572333 24572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:43.572362 24572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:43.572384 24572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:43.573611 24572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:43.573779 24572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:43.573994 24572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:43.578598 24580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.578619 24584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.578622 24581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:43.578723 24572 server_base.cc:1034] running on GCE node
I20260809 19:21:43.578953 24572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:43.579159 24572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:43.580281 24572 hybrid_clock.cc:648] HybridClock initialized: now 1786303303580275 us; error 26 us; skew 500 ppm
I20260809 19:21:43.581388 24572 webserver.cc:460] Webserver started at http://127.9.91.66:36443/ using document root <none> and password file <none>
I20260809 19:21:43.581593 24572 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:43.581634 24572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:43.581754 24572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:43.582701 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "8708a0d8e1cb47769d9115def1f5e433"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "80047e012f80a54fb31705edde269a97"
server_key_iv: "95773944530b414aabf18f5dcb470e09"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.583030 24572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "8708a0d8e1cb47769d9115def1f5e433"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "80047e012f80a54fb31705edde269a97"
server_key_iv: "95773944530b414aabf18f5dcb470e09"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.584208 24572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:21:43.584801 24592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.584941 24572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.585018 24572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "8708a0d8e1cb47769d9115def1f5e433"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "80047e012f80a54fb31705edde269a97"
server_key_iv: "95773944530b414aabf18f5dcb470e09"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.585081 24572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:43.588198 24572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:43.588418 24572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:43.588541 24572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:43.588722 24572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:43.588982 24572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:43.589015 24572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.589039 24572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:43.589056 24572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.594370 24572 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:41411
I20260809 19:21:43.594405 24789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:41411 every 8 connection(s)
I20260809 19:21:43.594679 24572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:21:43.597117 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.595304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:57017 (local address 127.9.91.126:46125)
0809 19:21:43.595351 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.595353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.595551 (+ 198us) server_negotiation.cc:407] Connection header received
0809 19:21:43.595932 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.595933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.595938 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.595977 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.596961 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:43.596962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:43.596962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:43.596964 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:43.596972 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:43.596975 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:43.597007 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:43.597051 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 19:21:43.597061 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:21:43.597280 24792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.595289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46125 (local address 127.9.91.66:57017)
0809 19:21:43.595454 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:43.595482 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:43.595851 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:43.596033 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:43.596036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:43.596043 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:43.596850 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:43.596851 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:43.596876 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:43.596901 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:43.597008 (+ 107us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:43.597011 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:43.597013 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:43.597057 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:21:43.597105 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260809 19:21:43.597450 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 24572
I20260809 19:21:43.597555 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:43.597697 24790 heartbeater.cc:347] Connected to a master server at 127.9.91.126:46125
I20260809 19:21:43.597751 9581 external_mini_cluster.cc:1428] Setting key aa2e542b05aa8f65993d2fc7f40cb0bd
I20260809 19:21:43.597780 24790 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:43.597919 24790 heartbeater.cc:511] Master 127.9.91.126:46125 requested a full tablet report, sending...
I20260809 19:21:43.598287 24256 ts_manager.cc:194] Registered new tserver with Master: 8708a0d8e1cb47769d9115def1f5e433 (127.9.91.66:41411)
I20260809 19:21:43.598712 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46125
--builtin_ntp_servers=127.9.91.84:35973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 19:21:43.677109 24793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:43.677276 24793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:43.677296 24793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:43.678360 24793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:43.678396 24793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:43.678419 24793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:43.679735 24793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:46125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:43.679910 24793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:43.680131 24793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:43.684517 24802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:43.684544 24805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:43.684602 24793 server_base.cc:1034] running on GCE node
W20260809 19:21:43.684509 24803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:43.684917 24793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:43.685153 24793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:43.686398 24793 hybrid_clock.cc:648] HybridClock initialized: now 1786303303686377 us; error 32 us; skew 500 ppm
I20260809 19:21:43.687602 24793 webserver.cc:460] Webserver started at http://127.9.91.67:42225/ using document root <none> and password file <none>
I20260809 19:21:43.687839 24793 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:43.687887 24793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:43.687983 24793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:43.689008 24793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "63e510a306f74f69bccfc54b7f944770"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "c1e6b8389bf1400a0350bf932e7f41e1"
server_key_iv: "f9f5a6362ae0669092f35e314f5696fd"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.689383 24793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "63e510a306f74f69bccfc54b7f944770"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "c1e6b8389bf1400a0350bf932e7f41e1"
server_key_iv: "f9f5a6362ae0669092f35e314f5696fd"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.690575 24793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:21:43.691289 24815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.691431 24793 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.691519 24793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "63e510a306f74f69bccfc54b7f944770"
format_stamp: "Formatted at 2026-08-09 19:21:43 on dist-test-slave-1vmg"
server_key: "c1e6b8389bf1400a0350bf932e7f41e1"
server_key_iv: "f9f5a6362ae0669092f35e314f5696fd"
server_key_version: "encryptionkey@0"
I20260809 19:21:43.691584 24793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:43.694790 24793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:43.695075 24793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:43.695211 24793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:43.695413 24793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:43.695701 24793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:43.695739 24793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.695767 24793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:43.695796 24793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:43.701776 24793 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:40861
I20260809 19:21:43.701807 24994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:40861 every 8 connection(s)
I20260809 19:21:43.702147 24793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:21:43.703388 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 24793
I20260809 19:21:43.703472 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:43.703636 9581 external_mini_cluster.cc:1428] Setting key ebcc9212b1db6a20297a95b904556bcb
I20260809 19:21:43.704933 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.702786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:59203 (local address 127.9.91.126:46125)
0809 19:21:43.702851 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.702853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.703061 (+ 208us) server_negotiation.cc:407] Connection header received
0809 19:21:43.703499 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.703501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.703509 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.703575 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.704739 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:43.704740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:43.704741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:21:43.704745 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:43.704762 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:21:43.704766 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:43.704822 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:43.704870 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 19:21:43.704882 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:21:43.705137 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:43.702731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46125 (local address 127.9.91.67:59203)
0809 19:21:43.702944 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:43.702983 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:43.703395 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:43.703639 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:43.703643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:43.703652 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:43.704619 (+ 967us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:21:43.704619 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:43.704644 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:21:43.704668 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:43.704822 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:43.704824 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:43.704826 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:21:43.704868 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 19:21:43.704908 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260809 19:21:43.705541 24996 heartbeater.cc:347] Connected to a master server at 127.9.91.126:46125
I20260809 19:21:43.705613 24996 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:43.705749 24996 heartbeater.cc:511] Master 127.9.91.126:46125 requested a full tablet report, sending...
I20260809 19:21:43.706061 24256 ts_manager.cc:194] Registered new tserver with Master: 63e510a306f74f69bccfc54b7f944770 (127.9.91.67:40861)
I20260809 19:21:43.715271 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 19:21:43.717001 24324 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:21:43.716578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.9.91.126:46125)
0809 19:21:43.716635 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:43.716637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:43.716736 (+ 99us) server_negotiation.cc:407] Connection header received
0809 19:21:43.716793 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:43.716794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:43.716798 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:43.716836 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:43.716948 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35806: BlockingRecv error: recv got EOF from 127.0.0.1:35806 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:21:43.717089 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 24357
I20260809 19:21:43.735989 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 24572
I20260809 19:21:43.749717 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 24793
I20260809 19:21:43.777701 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 24212
2026-08-09T19:21:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (577 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:43 dist-test-slave-1vmg krb5kdc[25015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:43 dist-test-slave-1vmg krb5kdc[25015](info): set up 2 sockets
Aug 09 19:21:43 dist-test-slave-1vmg krb5kdc[25015](info): commencing operation
krb5kdc: starting...
W20260809 19:21:44.896363 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:44 dist-test-slave-1vmg krb5kdc[25015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:21:45.003654 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36917
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:45.081733 25031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:45.081916 25031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:45.081947 25031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:45.083060 25031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:21:45.083098 25031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:45.083117 25031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:45.083133 25031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:21:45.083148 25031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:21:45.084573 25031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:45.084805 25031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:45.084993 25031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:45.089417 25044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:45.089409 25039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:45.089588 25031 server_base.cc:1034] running on GCE node
W20260809 19:21:45.089417 25040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:45.089845 25031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:45.090090 25031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:45.091220 25031 hybrid_clock.cc:648] HybridClock initialized: now 1786303305091202 us; error 33 us; skew 500 ppm
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:45.093473 25031 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:21:45.093876 25031 webserver.cc:460] Webserver started at http://127.9.91.126:45815/ using document root <none> and password file <none>
I20260809 19:21:45.094072 25031 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:45.094118 25031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:45.094240 25031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:45.095227 25031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "f30f2ae33c914dffafe067ea813dbda5"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "576d04edb12e053d506adf44576644e8"
server_key_iv: "56f4e1686348645e083174e6c1959f79"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.095533 25031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "f30f2ae33c914dffafe067ea813dbda5"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "576d04edb12e053d506adf44576644e8"
server_key_iv: "56f4e1686348645e083174e6c1959f79"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.096688 25031 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:45.097328 25053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.097460 25031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:45.097548 25031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "f30f2ae33c914dffafe067ea813dbda5"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "576d04edb12e053d506adf44576644e8"
server_key_iv: "56f4e1686348645e083174e6c1959f79"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.097611 25031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:45.116843 25031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:45.118369 25031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:45.118485 25031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:45.122064 25031 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:36917
I20260809 19:21:45.122107 25140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:36917 every 8 connection(s)
I20260809 19:21:45.122372 25031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:21:45.123250 25141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:45.125298 25141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5: Bootstrap starting.
I20260809 19:21:45.125806 25141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:45.126029 25141 log.cc:824] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:45.126616 25141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5: No bootstrap required, opened a new log
I20260809 19:21:45.127641 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 25031
I20260809 19:21:45.127741 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:21:45.127923 9581 external_mini_cluster.cc:1428] Setting key 7d472ec79b042f177a40f56e7d4c6ec2
I20260809 19:21:45.128496 25141 raft_consensus.cc:348] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2ae33c914dffafe067ea813dbda5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36917 } }
I20260809 19:21:45.128644 25141 raft_consensus.cc:374] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:45.128679 25141 raft_consensus.cc:729] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30f2ae33c914dffafe067ea813dbda5, State: Initialized, Role: FOLLOWER
I20260809 19:21:45.128798 25141 consensus_queue.cc:260] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2ae33c914dffafe067ea813dbda5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36917 } }
I20260809 19:21:45.128870 25141 raft_consensus.cc:388] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:21:45.128901 25141 raft_consensus.cc:482] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:21:45.128978 25141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:45.129628 25141 raft_consensus.cc:504] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2ae33c914dffafe067ea813dbda5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36917 } }
I20260809 19:21:45.129735 25141 leader_election.cc:302] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f30f2ae33c914dffafe067ea813dbda5; no voters:
I20260809 19:21:45.129892 25141 leader_election.cc:288] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:21:45.129941 25150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:45.130084 25150 raft_consensus.cc:686] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [term 1 LEADER]: Becoming Leader. State: Replica: f30f2ae33c914dffafe067ea813dbda5, State: Running, Role: LEADER
I20260809 19:21:45.130165 25141 sys_catalog.cc:564] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:21:45.130232 25150 consensus_queue.cc:237] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2ae33c914dffafe067ea813dbda5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36917 } }
I20260809 19:21:45.130750 25150 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f30f2ae33c914dffafe067ea813dbda5. Latest consensus state: current_term: 1 leader_uuid: "f30f2ae33c914dffafe067ea813dbda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2ae33c914dffafe067ea813dbda5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36917 } } }
I20260809 19:21:45.130638 25152 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f30f2ae33c914dffafe067ea813dbda5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30f2ae33c914dffafe067ea813dbda5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36917 } } }
I20260809 19:21:45.130810 25150 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:45.130860 25152 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:21:45.131153 25168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:21:45.131651 25168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:21:45.133229 25168 catalog_manager.cc:1347] Generated new cluster ID: 05862442c8df4dcdb2504b787c834ae6
I20260809 19:21:45.133279 25168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:45.138664 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.128421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.9.91.126:36917)
0809 19:21:45.128570 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.128577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.128599 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:21:45.128698 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.128704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.128757 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.128881 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:45.129603 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.131073 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.132025 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.132571 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.132676 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.135214 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:45.135217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:45.135218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:45.135229 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:45.137056 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.137411 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.137414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.137414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.137542 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.137713 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.137713 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.137714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.137999 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:45.138159 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.138227 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.138361 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260809 19:21:45.141382 25168 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:21:45.142045 25168 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:21:45.147217 25168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5: Generated new TSK 0
I20260809 19:21:45.147362 25168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:21:45.173692 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36917
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:45.252395 25190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:45.252575 25190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:45.252596 25190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:45.253683 25190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:45.253723 25190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:45.253750 25190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:21:45.255187 25190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:45.255402 25190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:45.255600 25190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:45.260079 25202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:45.260126 25198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:45.260197 25199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:45.260411 25190 server_base.cc:1034] running on GCE node
I20260809 19:21:45.260641 25190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:45.260859 25190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:45.262003 25190 hybrid_clock.cc:648] HybridClock initialized: now 1786303305261987 us; error 32 us; skew 500 ppm
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:45.264442 25190 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:21:45.264809 25190 webserver.cc:460] Webserver started at http://127.9.91.65:40175/ using document root <none> and password file <none>
I20260809 19:21:45.264984 25190 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:45.265022 25190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:45.265111 25190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:45.266078 25190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "13c2f18d97514845b0217e0385660468"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "9d652ba6112e66debc88105b119f60bc"
server_key_iv: "16e94e05e0d9fc13ba38711a1dc54e24"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.266359 25190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "13c2f18d97514845b0217e0385660468"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "9d652ba6112e66debc88105b119f60bc"
server_key_iv: "16e94e05e0d9fc13ba38711a1dc54e24"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.267581 25190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:21:45.268181 25218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.268306 25190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:45.268385 25190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "13c2f18d97514845b0217e0385660468"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "9d652ba6112e66debc88105b119f60bc"
server_key_iv: "16e94e05e0d9fc13ba38711a1dc54e24"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.268450 25190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:45.280345 25190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:45.282214 25190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:45.282336 25190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:45.282512 25190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:45.282783 25190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:45.282821 25190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.282840 25190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:45.282861 25190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.288286 25190 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:45179
I20260809 19:21:45.288354 25398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:45179 every 8 connection(s)
I20260809 19:21:45.288597 25190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:45.296172 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.289182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:55355 (local address 127.9.91.126:36917)
0809 19:21:45.289231 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.289233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.289428 (+ 195us) server_negotiation.cc:407] Connection header received
0809 19:21:45.289837 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.289838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.289848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.289893 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:45.290697 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.291225 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.292127 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.292308 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.292349 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.294123 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:45.294125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:45.294125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:45.294133 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:45.295219 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.295540 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.295541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.295541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.295582 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.295732 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.295733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.295733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.295818 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:45.295861 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.296047 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.296112 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:21:45.296326 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.289150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36917 (local address 127.9.91.65:55355)
0809 19:21:45.289308 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.289346 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.289763 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.289946 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.289949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.290454 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:45.290631 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.290635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.291292 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.291292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.292066 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.292068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.292362 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.292362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.292452 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.292800 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:45.292808 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:45.294026 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:45.295282 (+ 1256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:45.295289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:45.295290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:45.295484 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:45.295616 (+ 132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:45.295617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:45.295617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:45.295695 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:45.295859 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:45.295862 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:45.295959 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.296030 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.296148 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260809 19:21:45.296749 25399 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36917
I20260809 19:21:45.296825 25399 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:45.296955 25399 heartbeater.cc:511] Master 127.9.91.126:36917 requested a full tablet report, sending...
I20260809 19:21:45.297398 25084 ts_manager.cc:194] Registered new tserver with Master: 13c2f18d97514845b0217e0385660468 (127.9.91.65:45179)
I20260809 19:21:45.297842 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 25190
I20260809 19:21:45.297945 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:21:45.298105 9581 external_mini_cluster.cc:1428] Setting key b74f018c3b044cf496a23a713bb54a96
I20260809 19:21:45.298187 25084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:55355
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:21:45.320806 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36917
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:45.401733 25410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:45.401912 25410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:45.401942 25410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:45.403069 25410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:45.403108 25410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:45.403142 25410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:21:45.404587 25410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:45.404810 25410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:45.405005 25410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:45.409411 25419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:45.409559 25410 server_base.cc:1034] running on GCE node
W20260809 19:21:45.409421 25417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:45.409412 25416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:45.409840 25410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:45.410063 25410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:45.411203 25410 hybrid_clock.cc:648] HybridClock initialized: now 1786303305411190 us; error 27 us; skew 500 ppm
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:45.413419 25410 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:21:45.413828 25410 webserver.cc:460] Webserver started at http://127.9.91.66:38691/ using document root <none> and password file <none>
I20260809 19:21:45.414024 25410 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:45.414070 25410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:45.414192 25410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:45.415199 25410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "468d1907e5a944a78d84e71e0029c2a3"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "428399224ee22a5f5c0b18a0bec3a428"
server_key_iv: "6e887657ac57cea7ab5ecdde2754e394"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.415513 25410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "468d1907e5a944a78d84e71e0029c2a3"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "428399224ee22a5f5c0b18a0bec3a428"
server_key_iv: "6e887657ac57cea7ab5ecdde2754e394"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.416667 25410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:21:45.417413 25432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.417590 25410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:21:45.417681 25410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "468d1907e5a944a78d84e71e0029c2a3"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "428399224ee22a5f5c0b18a0bec3a428"
server_key_iv: "6e887657ac57cea7ab5ecdde2754e394"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.417747 25410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:45.439129 25410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:45.440699 25410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:45.440815 25410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:45.441007 25410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:45.441283 25410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:45.441318 25410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.441335 25410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:45.441357 25410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.446691 25410 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:44307
I20260809 19:21:45.446760 25607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:44307 every 8 connection(s)
I20260809 19:21:45.447019 25410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:45.454802 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.447653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:47729 (local address 127.9.91.126:36917)
0809 19:21:45.447709 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.447711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.447878 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:21:45.448333 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.448334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.448347 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.448394 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:45.449254 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.449748 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.450670 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.450868 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.450941 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.452777 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:45.452778 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:45.452779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:45.452784 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:45.453822 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.454148 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.454149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.454149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.454193 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.454363 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.454363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.454363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.454449 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:45.454491 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.454683 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.454739 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:21:45.455000 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.447634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36917 (local address 127.9.91.66:47729)
0809 19:21:45.447773 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.447803 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.448236 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.448465 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.448469 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.448982 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:45.449159 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.449163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.449830 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.449831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.450583 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.450586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.450958 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.450959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.451064 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.451441 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:45.451448 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:45.452690 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:45.453882 (+ 1192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:45.453888 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:45.453889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:45.454092 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:45.454236 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:45.454236 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:45.454237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:45.454322 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:45.454489 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:45.454491 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:45.454595 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.454668 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.454772 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260809 19:21:45.455019 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 25410
I20260809 19:21:45.455117 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:21:45.455310 9581 external_mini_cluster.cc:1428] Setting key 68a9b30864c800757621328a94e98e02
I20260809 19:21:45.455436 25608 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36917
I20260809 19:21:45.455531 25608 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:45.455691 25608 heartbeater.cc:511] Master 127.9.91.126:36917 requested a full tablet report, sending...
I20260809 19:21:45.456010 25084 ts_manager.cc:194] Registered new tserver with Master: 468d1907e5a944a78d84e71e0029c2a3 (127.9.91.66:44307)
I20260809 19:21:45.456357 25084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:47729
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:21:45.477342 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36917
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:21:45.555953 25615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:21:45.556159 25615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:21:45.556193 25615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:21:45.557301 25615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:21:45.557348 25615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:21:45.557384 25615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:21:45.558805 25615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:21:45.559075 25615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:21:45.559276 25615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:21:45.564035 25620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:45.564038 25624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:21:45.564121 25622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:21:45.564170 25615 server_base.cc:1034] running on GCE node
I20260809 19:21:45.564466 25615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:21:45.564692 25615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:21:45.565830 25615 hybrid_clock.cc:648] HybridClock initialized: now 1786303305565818 us; error 24 us; skew 500 ppm
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:21:45.568194 25615 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:21:45.568619 25615 webserver.cc:460] Webserver started at http://127.9.91.67:38827/ using document root <none> and password file <none>
I20260809 19:21:45.568827 25615 fs_manager.cc:359] Metadata directory not provided
I20260809 19:21:45.568879 25615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:21:45.569020 25615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:21:45.569981 25615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "ad3a443a9ef54f8e881af6a7947dce87"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "f2e251f81b84c62be3f08f7ad278bf67"
server_key_iv: "9e2026dc9a5b18aebd1498a81d2ad8bc"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.570333 25615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "ad3a443a9ef54f8e881af6a7947dce87"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "f2e251f81b84c62be3f08f7ad278bf67"
server_key_iv: "9e2026dc9a5b18aebd1498a81d2ad8bc"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.571771 25615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:21:45.572494 25637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.572645 25615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:21:45.572732 25615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "ad3a443a9ef54f8e881af6a7947dce87"
format_stamp: "Formatted at 2026-08-09 19:21:45 on dist-test-slave-1vmg"
server_key: "f2e251f81b84c62be3f08f7ad278bf67"
server_key_iv: "9e2026dc9a5b18aebd1498a81d2ad8bc"
server_key_version: "encryptionkey@0"
I20260809 19:21:45.572796 25615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:21:45.583408 25615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:21:45.585017 25615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:21:45.585150 25615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:21:45.585343 25615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:21:45.585621 25615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:21:45.585659 25615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.585687 25615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:21:45.585716 25615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:21:45.591849 25615 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:37577
I20260809 19:21:45.591886 25806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:37577 every 8 connection(s)
I20260809 19:21:45.592170 25615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25015](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303305, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:21:45.600543 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.592777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:54229 (local address 127.9.91.126:36917)
0809 19:21:45.592844 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.592846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.593016 (+ 170us) server_negotiation.cc:407] Connection header received
0809 19:21:45.593472 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.593473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.593487 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.593561 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:45.594438 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.594977 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.595932 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.596144 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.596211 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.598070 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:45.598071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:45.598072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:45.598079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:45.599221 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.599598 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.599599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.599599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.599659 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.599903 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.599905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.599905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.600025 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:45.600090 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.600350 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.600435 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:21:45.600660 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.592739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36917 (local address 127.9.91.67:54229)
0809 19:21:45.592903 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.592937 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.593372 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.593630 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.593633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.594158 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:21:45.594334 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.594339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.595074 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.595075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.595840 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.595843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.596220 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.596221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.596342 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.596710 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:21:45.596717 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:21:45.597961 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:21:45.599293 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:45.599301 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:45.599303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:45.599521 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:45.599718 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:21:45.599719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:21:45.599719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:21:45.599840 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:21:45.600088 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:21:45.600095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:21:45.600227 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.600318 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.600462 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260809 19:21:45.601087 25808 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36917
I20260809 19:21:45.601219 25808 heartbeater.cc:464] Registering TS with master...
I20260809 19:21:45.601333 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 25615
I20260809 19:21:45.601411 25808 heartbeater.cc:511] Master 127.9.91.126:36917 requested a full tablet report, sending...
I20260809 19:21:45.601439 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:21:45.601608 9581 external_mini_cluster.cc:1428] Setting key d8c87bd231aeec01c9daa550f852954d
I20260809 19:21:45.601814 25084 ts_manager.cc:194] Registered new tserver with Master: ad3a443a9ef54f8e881af6a7947dce87 (127.9.91.67:37577)
I20260809 19:21:45.602264 25084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:54229
I20260809 19:21:45.602497 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:21:45.608839 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.603809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45366 (local address 127.9.91.126:36917)
0809 19:21:45.603857 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.603859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.603980 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:21:45.604039 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.604040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.604047 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.604091 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:21:45.604618 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.605041 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.605692 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.605854 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.605895 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.606890 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:21:45.606892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:21:45.606892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:21:45.606897 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:21:45.608001 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.608298 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.608299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.608299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.608342 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:21:45.608464 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:21:45.608465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:21:45.608465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:21:45.608575 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:21:45.608644 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.608732 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.608784 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:21:45.610142 25084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:21:45.611727 25084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:21:45.617693 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.613479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:37577 (local address 127.0.0.1:51298)
0809 19:21:45.614019 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.614073 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.614145 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.614395 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.614398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.614401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:45.614458 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.614460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.615657 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.615658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.616756 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.616759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.617424 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.617425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.617500 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.617512 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.617575 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.617616 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":82,"threads_started":1}
I20260809 19:21:45.617702 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.613621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51298 (local address 127.9.91.67:37577)
0809 19:21:45.613794 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.613798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.614143 (+ 345us) server_negotiation.cc:407] Connection header received
0809 19:21:45.614237 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.614239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.614254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.614352 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:45.614533 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.615549 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.616830 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.617341 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.617425 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.617478 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.617588 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.617623 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260809 19:21:45.618188 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.613484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:44307 (local address 127.0.0.1:43630)
0809 19:21:45.614262 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.614272 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.614327 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.614933 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.614935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.614938 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:45.614999 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.615001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.616056 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.616057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.617098 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.617100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.617950 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.617951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.618015 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.618019 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.618081 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.618119 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":83,"threads_started":1}
I20260809 19:21:45.618213 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.613791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43630 (local address 127.9.91.66:44307)
0809 19:21:45.613965 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.613969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.614376 (+ 407us) server_negotiation.cc:407] Connection header received
0809 19:21:45.614725 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.614728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.614746 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.614827 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:45.615085 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.615957 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.617182 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.617862 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.617936 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.618001 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.618096 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.618132 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260809 19:21:45.618568 25711 tablet_service.cc:1467] Processing CreateTablet for tablet 9dabd53e80ef4388ab7d8a9b6521ea94 (DEFAULT_TABLE table=test-table [id=331429af71ea42218afbb5cb7a25d089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:45.618897 25711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dabd53e80ef4388ab7d8a9b6521ea94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:45.619027 25502 tablet_service.cc:1467] Processing CreateTablet for tablet 9dabd53e80ef4388ab7d8a9b6521ea94 (DEFAULT_TABLE table=test-table [id=331429af71ea42218afbb5cb7a25d089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:45.619217 25818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.613479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:45179 (local address 127.0.0.1:41838)
0809 19:21:45.614516 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.614525 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.614580 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.614768 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.614769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.614771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:45.614816 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.614817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.616849 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.616850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.617738 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.617742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.618960 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.618961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.619035 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.619040 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.619109 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.619148 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"mutex_wait_us":8,"spinlock_wait_cycles":384,"thread_start_us":92,"threads_started":1}
I20260809 19:21:45.619259 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.613769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.9.91.65:45179)
0809 19:21:45.613921 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.613925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.614578 (+ 653us) server_negotiation.cc:407] Connection header received
0809 19:21:45.614665 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.614675 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.614691 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.614762 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:45.614879 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.615567 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.617834 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.618530 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.618602 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.618658 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.619130 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.619180 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260809 19:21:45.619374 25502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dabd53e80ef4388ab7d8a9b6521ea94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:45.619920 25295 tablet_service.cc:1467] Processing CreateTablet for tablet 9dabd53e80ef4388ab7d8a9b6521ea94 (DEFAULT_TABLE table=test-table [id=331429af71ea42218afbb5cb7a25d089]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:21:45.620278 25295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dabd53e80ef4388ab7d8a9b6521ea94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:21:45.621493 25824 tablet_bootstrap.cc:492] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Bootstrap starting.
I20260809 19:21:45.622103 25824 tablet_bootstrap.cc:654] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:45.622498 25824 log.cc:824] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:45.623165 25824 tablet_bootstrap.cc:492] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: No bootstrap required, opened a new log
I20260809 19:21:45.623225 25824 ts_tablet_manager.cc:1397] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:21:45.623657 25832 tablet_bootstrap.cc:492] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468: Bootstrap starting.
I20260809 19:21:45.624130 25827 tablet_bootstrap.cc:492] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Bootstrap starting.
I20260809 19:21:45.624377 25832 tablet_bootstrap.cc:654] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:45.624675 25824 raft_consensus.cc:348] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.624796 25824 raft_consensus.cc:374] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:45.624828 25824 raft_consensus.cc:729] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3a443a9ef54f8e881af6a7947dce87, State: Initialized, Role: FOLLOWER
I20260809 19:21:45.624856 25827 tablet_bootstrap.cc:654] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Neither blocks nor log segments found. Creating new log.
I20260809 19:21:45.624949 25832 log.cc:824] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:45.624975 25824 consensus_queue.cc:260] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.625283 25824 ts_tablet_manager.cc:1428] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:21:45.625291 25808 heartbeater.cc:503] Master 127.9.91.126:36917 was elected leader, sending a full tablet report...
I20260809 19:21:45.625468 25827 log.cc:824] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Log is configured to *not* fsync() on all Append() calls
I20260809 19:21:45.626279 25832 tablet_bootstrap.cc:492] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468: No bootstrap required, opened a new log
I20260809 19:21:45.626348 25827 tablet_bootstrap.cc:492] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: No bootstrap required, opened a new log
I20260809 19:21:45.626394 25832 ts_tablet_manager.cc:1397] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:21:45.626406 25827 ts_tablet_manager.cc:1397] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:21:45.628412 25827 raft_consensus.cc:348] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.628412 25832 raft_consensus.cc:348] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.628582 25827 raft_consensus.cc:374] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:45.628582 25832 raft_consensus.cc:374] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:21:45.628629 25827 raft_consensus.cc:729] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468d1907e5a944a78d84e71e0029c2a3, State: Initialized, Role: FOLLOWER
I20260809 19:21:45.628629 25832 raft_consensus.cc:729] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c2f18d97514845b0217e0385660468, State: Initialized, Role: FOLLOWER
I20260809 19:21:45.628744 25832 consensus_queue.cc:260] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.628744 25827 consensus_queue.cc:260] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.629035 25832 ts_tablet_manager.cc:1428] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:21:45.629038 25827 ts_tablet_manager.cc:1428] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:21:45.629051 25608 heartbeater.cc:503] Master 127.9.91.126:36917 was elected leader, sending a full tablet report...
I20260809 19:21:45.629129 25399 heartbeater.cc:503] Master 127.9.91.126:36917 was elected leader, sending a full tablet report...
I20260809 19:21:45.696015 25838 raft_consensus.cc:482] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:21:45.696118 25838 raft_consensus.cc:504] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.696471 25838 leader_election.cc:288] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad3a443a9ef54f8e881af6a7947dce87 (127.9.91.67:37577), 13c2f18d97514845b0217e0385660468 (127.9.91.65:45179)
W20260809 19:21:45.697537 25610 tablet.cc:2369] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:21:45.699771 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.696680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:54071 (local address 127.9.91.67:37577)
0809 19:21:45.696718 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.696719 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.696755 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:21:45.696780 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.696781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.696791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.696841 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:45.697009 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.697502 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.698827 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.699406 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.699480 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.699491 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.699633 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.699694 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:21:45.699781 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.696669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:36247 (local address 127.9.91.65:45179)
0809 19:21:45.696713 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.696716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.696860 (+ 144us) server_negotiation.cc:407] Connection header received
0809 19:21:45.696922 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.696923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.696936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.696978 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:21:45.697140 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.697649 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.698930 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.699473 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.699534 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.699544 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.699676 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.699713 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:21:45.699766 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.696608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:37577 (local address 127.9.91.66:54071)
0809 19:21:45.696663 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.696673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.696713 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.696863 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.696864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.696865 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:45.696948 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.696950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.697567 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.697569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.698740 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.698743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.699476 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.699477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.699567 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.699571 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.699625 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.699668 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":21}
I20260809 19:21:45.699771 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.696608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:45179 (local address 127.9.91.66:36247)
0809 19:21:45.696787 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.696798 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:21:45.696858 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:21:45.696994 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:21:45.696995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:21:45.696996 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:21:45.697080 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.697082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.697736 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.697737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.698846 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:21:45.698848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.699537 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.699538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:21:45.699603 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.699606 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:21:45.699650 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:21:45.699680 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":12,"thread_start_us":105,"threads_started":1}
I20260809 19:21:45.700114 25329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dabd53e80ef4388ab7d8a9b6521ea94" candidate_uuid: "468d1907e5a944a78d84e71e0029c2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c2f18d97514845b0217e0385660468" is_pre_election: true
I20260809 19:21:45.700273 25329 raft_consensus.cc:2454] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468d1907e5a944a78d84e71e0029c2a3 in term 0.
I20260809 19:21:45.700235 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dabd53e80ef4388ab7d8a9b6521ea94" candidate_uuid: "468d1907e5a944a78d84e71e0029c2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3a443a9ef54f8e881af6a7947dce87" is_pre_election: true
I20260809 19:21:45.700395 25745 raft_consensus.cc:2454] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 468d1907e5a944a78d84e71e0029c2a3 in term 0.
I20260809 19:21:45.700461 25434 leader_election.cc:302] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c2f18d97514845b0217e0385660468, 468d1907e5a944a78d84e71e0029c2a3; no voters:
I20260809 19:21:45.700598 25838 raft_consensus.cc:2781] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:21:45.700678 25838 raft_consensus.cc:482] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:21:45.700703 25838 raft_consensus.cc:3037] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:45.701376 25838 raft_consensus.cc:504] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.701481 25838 leader_election.cc:288] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [CANDIDATE]: Term 1 election: Requested vote from peers ad3a443a9ef54f8e881af6a7947dce87 (127.9.91.67:37577), 13c2f18d97514845b0217e0385660468 (127.9.91.65:45179)
I20260809 19:21:45.701618 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dabd53e80ef4388ab7d8a9b6521ea94" candidate_uuid: "468d1907e5a944a78d84e71e0029c2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad3a443a9ef54f8e881af6a7947dce87"
I20260809 19:21:45.701640 25329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dabd53e80ef4388ab7d8a9b6521ea94" candidate_uuid: "468d1907e5a944a78d84e71e0029c2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c2f18d97514845b0217e0385660468"
I20260809 19:21:45.701687 25745 raft_consensus.cc:3037] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:45.701704 25329 raft_consensus.cc:3037] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:21:45.702363 25745 raft_consensus.cc:2454] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468d1907e5a944a78d84e71e0029c2a3 in term 1.
I20260809 19:21:45.702366 25329 raft_consensus.cc:2454] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 468d1907e5a944a78d84e71e0029c2a3 in term 1.
I20260809 19:21:45.702539 25437 leader_election.cc:302] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 468d1907e5a944a78d84e71e0029c2a3, ad3a443a9ef54f8e881af6a7947dce87; no voters:
I20260809 19:21:45.702644 25838 raft_consensus.cc:2781] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:21:45.702760 25838 raft_consensus.cc:686] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 1 LEADER]: Becoming Leader. State: Replica: 468d1907e5a944a78d84e71e0029c2a3, State: Running, Role: LEADER
I20260809 19:21:45.702844 25838 consensus_queue.cc:237] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } }
I20260809 19:21:45.703585 25084 catalog_manager.cc:5495] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 468d1907e5a944a78d84e71e0029c2a3 (127.9.91.66). New cstate: current_term: 1 leader_uuid: "468d1907e5a944a78d84e71e0029c2a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "468d1907e5a944a78d84e71e0029c2a3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 } health_report { overall_health: UNKNOWN } } }
I20260809 19:21:45.711681 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.709457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.9.91.66:44307)
0809 19:21:45.709494 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:21:45.709496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:21:45.709503 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:21:45.709562 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:21:45.709563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:21:45.709581 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:21:45.709642 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:21:45.709764 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.710190 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.710874 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:21:45.711165 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:21:45.711221 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:21:45.711375 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:21:45.711469 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:21:45.711535 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:21:45.711573 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 19:21:45.711592 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:21:45.713025 25329 raft_consensus.cc:1261] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 13c2f18d97514845b0217e0385660468 [term 1 FOLLOWER]: Refusing update from remote peer 468d1907e5a944a78d84e71e0029c2a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:45.713047 25745 raft_consensus.cc:1261] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 1 FOLLOWER]: Refusing update from remote peer 468d1907e5a944a78d84e71e0029c2a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:21:45.713296 25838 consensus_queue.cc:1035] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c2f18d97514845b0217e0385660468" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 45179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:45.713420 25840 consensus_queue.cc:1035] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad3a443a9ef54f8e881af6a7947dce87" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:21:45.715787 25844 mvcc.cc:205] Tried to move back new op lower bound from 7316698340198682624 to 7316698340159483904. Current Snapshot: MvccSnapshot[applied={T|T < 7316698340198682624}]
I20260809 19:21:45.716147 25845 mvcc.cc:205] Tried to move back new op lower bound from 7316698340198682624 to 7316698340159483904. Current Snapshot: MvccSnapshot[applied={T|T < 7316698340198682624}]
I20260809 19:21:45.717473 25084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:45.717550 25084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:21:45.718163 25084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f30f2ae33c914dffafe067ea813dbda5: Sending DeleteTablet for 3 replicas of tablet 9dabd53e80ef4388ab7d8a9b6521ea94
I20260809 19:21:45.718485 25295 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dabd53e80ef4388ab7d8a9b6521ea94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:45 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:41838
I20260809 19:21:45.718488 25711 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dabd53e80ef4388ab7d8a9b6521ea94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:45 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:51298
I20260809 19:21:45.718719 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 25190
I20260809 19:21:45.718762 25502 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dabd53e80ef4388ab7d8a9b6521ea94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:21:45 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:43630
I20260809 19:21:45.721278 25858 tablet_replica.cc:330] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: stopping tablet replica
I20260809 19:21:45.721400 25858 raft_consensus.cc:2229] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:21:45.721516 25858 raft_consensus.cc:2258] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:45.722860 25858 ts_tablet_manager.cc:1905] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:45.724470 25858 ts_tablet_manager.cc:1918] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:45.724545 25858 log.cc:1196] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/9dabd53e80ef4388ab7d8a9b6521ea94
I20260809 19:21:45.724776 25858 ts_tablet_manager.cc:1939] T 9dabd53e80ef4388ab7d8a9b6521ea94 P ad3a443a9ef54f8e881af6a7947dce87: Deleting consensus metadata
I20260809 19:21:45.725386 25058 catalog_manager.cc:4841] TS ad3a443a9ef54f8e881af6a7947dce87 (127.9.91.67:37577): tablet 9dabd53e80ef4388ab7d8a9b6521ea94 (table test-table [id=331429af71ea42218afbb5cb7a25d089]) successfully deleted
I20260809 19:21:45.736210 25863 tablet_replica.cc:330] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: stopping tablet replica
I20260809 19:21:45.736315 25863 raft_consensus.cc:2229] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:21:45.736397 25863 raft_consensus.cc:2258] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:21:45.736909 25863 ts_tablet_manager.cc:1905] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:21:45.738152 25863 ts_tablet_manager.cc:1918] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:21:45.738224 25863 log.cc:1196] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/9dabd53e80ef4388ab7d8a9b6521ea94
I20260809 19:21:45.738493 25863 ts_tablet_manager.cc:1939] T 9dabd53e80ef4388ab7d8a9b6521ea94 P 468d1907e5a944a78d84e71e0029c2a3: Deleting consensus metadata
I20260809 19:21:45.738873 25054 catalog_manager.cc:4841] TS 468d1907e5a944a78d84e71e0029c2a3 (127.9.91.66:44307): tablet 9dabd53e80ef4388ab7d8a9b6521ea94 (table test-table [id=331429af71ea42218afbb5cb7a25d089]) successfully deleted
W20260809 19:21:45.741708 25055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:45179 (error 108)
I20260809 19:21:45.741868 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 25410
I20260809 19:21:45.745956 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:21:45.745823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:45179 (local address 127.0.0.1:41840)
0809 19:21:45.745887 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 19:21:45.745930 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 19:21:45.746723 25055 catalog_manager.cc:4584] TS 13c2f18d97514845b0217e0385660468 (127.9.91.65:45179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9dabd53e80ef4388ab7d8a9b6521ea94: Network error: Client connection negotiation failed: client connection to 127.9.91.65:45179: connect: Connection refused (error 111)
I20260809 19:21:45.751263 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 25615
I20260809 19:21:45.758759 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 25031
2026-08-09T19:21:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1981 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25867](info): set up 2 sockets
Aug 09 19:21:45 dist-test-slave-1vmg krb5kdc[25867](info): commencing operation
krb5kdc: starting...
W20260809 19:21:46.886488 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:21:46 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:21:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:21:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/rangeradmin_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/rangeradmin_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/rangerlookup_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/rangerlookup_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
I20260809 19:21:47.031132 9581 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskt8E36c/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/postgres -l logfile start
2026-08-09 19:21:49.196 UTC [25912] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 19:21:49.196 UTC [25912] LOG: listening on IPv4 address "127.9.91.84", port 45147
2026-08-09 19:21:49.199 UTC [25912] LOG: listening on Unix socket "/tmp/.s.PGSQL.45147"
2026-08-09 19:21:49.219 UTC [25915] LOG: database system was shut down at 2026-08-09 19:21:48 UTC
2026-08-09 19:21:49.222 UTC [25912] LOG: database system is ready to accept connections
I20260809 19:21:50.262931 9581 mini_postgres.cc:96] Postgres bound to 45147
2026-08-09 19:21:50.270 UTC [25929] FATAL: database "slave" does not exist
127.9.91.84:45147 - accepting connections
I20260809 19:21:50.271144 9581 mini_ranger.cc:161] Starting Ranger...
I20260809 19:21:50.286619 9581 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 19:21:50.664057 9581 mini_ranger.cc:87] Created ranger Postgres database
I20260809 19:21:50.664142 9581 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-admin
2026-08-09 19:21:51,039 [I] DB FLAVOR :POSTGRES
2026-08-09 19:21:51,039 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:21:51,039 [I] Checking connection..
2026-08-09 19:21:51,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 19:21:51,305 [I] Checking connection passed.
2026-08-09 19:21:51,305 [I] --------- Verifying version history table ---------
2026-08-09 19:21:51,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:21:51,570 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:21:51,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:21:51,803 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:21:51,803 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 19:21:51,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 19:21:52.054 UTC [26030] WARNING: there is no transaction in progress
2026-08-09 19:21:52,062 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 19:21:52,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:21:52,295 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 19:21:52,295 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:21:52,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 19:21:52,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 19:21:52,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1vmg.c.gcp-upstream.internal','N') ;"
2026-08-09 19:21:53,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:21:53,278 [I] Table x_portal_user does not exist in database ranger
2026-08-09 19:21:53,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:21:53,510 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 19:21:53,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:21:53,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:21:53,980 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 19:21:53,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 19:21:54.821 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.836 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.850 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.863 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.876 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.894 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.942 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.947 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.960 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.969 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.978 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.990 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:54.998 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:55.004 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:55.010 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:55.016 UTC [26201] WARNING: there is no transaction in progress
2026-08-09 19:21:55,556 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 19:21:55,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 19:21:55,782 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 19:21:55,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:21:56,022 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 19:21:56,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:21:56,261 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 19:21:56,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:21:56,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:21:56,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1vmg.c.gcp-upstream.internal';"
2026-08-09 19:21:56,984 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 19:21:56,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 19:21:57,226 [I] DB_PATCHES have already been applied
I20260809 19:21:57.232309 9581 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-admin:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskt8E36c/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 19:21:57.232366 9581 mini_ranger.cc:193] Using host: 127.9.91.84
I20260809 19:21:57.235857 9581 mini_ranger.cc:227] Ranger admin URL: http://127.9.91.84:55853
Aug 09, 2026 7:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:21:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:21:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:21:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 19:21:58.464125 9581 mini_ranger.cc:160] Time spent starting Ranger: real 8.193s user 0.004s sys 0.006s
Aug 09, 2026 7:22:18 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:22:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 7:22:18 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:22:18 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:22:19 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 19:22:20.081378 9581 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/rangerkms_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/rangerkms_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.91.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.91.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 19:22:20 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303340, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 19:22:20.174412 9581 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 19:22:20.184377 9581 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 19:22:20.852119 9581 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 19:22:20.852212 9581 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms
I20260809 19:22:20.852231 9581 mini_ranger_kms.cc:227] Using postgres at 127.9.91.84:45147
2026-08-09 19:22:21,160 [I] DB FLAVOR :POSTGRES
2026-08-09 19:22:21,160 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:22:21,160 [I] Checking connection
2026-08-09 19:22:21,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 19:22:21,405 [I] connection success
2026-08-09 19:22:21,405 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 19:22:21,405 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 19:22:21,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 19:22:21,635 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 19:22:21,635 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:22:21,635 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 19:22:21,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:45147/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 19:22:21,907 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 19:22:22.309423 9581 mini_ranger_kms.cc:310] Created kms service
I20260809 19:22:22.502166 9581 mini_ranger_kms.cc:326] Created kudu user
I20260809 19:22:22.548609 9581 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 19:22:22 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303340, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:22:22.934516 9581 mini_ranger_kms.cc:384] Added ranger policy
I20260809 19:22:22.934633 9581 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms:/tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 19:22:22.934657 9581 mini_ranger_kms.cc:240] Using host: 127.9.91.84
I20260809 19:22:22.938763 9581 mini_ranger_kms.cc:276] Ranger KMS PID: 26479
I20260809 19:22:22.938895 9581 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.91.84:50183
Aug 09, 2026 7:22:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:22:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:22:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:22:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:22:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:22:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 19:22:24 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.91.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:22:25.413002 9581 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.239s user 0.004s sys 0.247s
I20260809 19:22:25.413122 9581 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 19:22:25.413211 9581 mini_ranger_kms.cc:401] 127.9.91.84:50183/kms/v1/keys
Aug 09 19:22:25 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.91.84@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
Aug 09 19:22:25 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.91.84@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
Aug 09 19:22:28 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:22:28.123114 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35549
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:35549
--ranger_config_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:50183/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:22:28.202416 26554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:22:28.202595 26554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:22:28.202615 26554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:22:28.203747 26554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:22:28.203786 26554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:22:28.203799 26554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:22:28.203810 26554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:22:28.203821 26554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:22:28.205353 26554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.91.84:50183/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:35549
--ranger_config_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:22:28.205552 26554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:22:28.205760 26554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:22:28.210383 26561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:22:28.210465 26554 server_base.cc:1034] running on GCE node
W20260809 19:22:28.210391 26566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:22:28.210484 26569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:22:28.210884 26554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:22:28.211207 26554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:22:28.212348 26554 hybrid_clock.cc:648] HybridClock initialized: now 1786303348212329 us; error 37 us; skew 500 ppm
Aug 09 19:22:28 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:22:28.217424 26554 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:22:28.217908 26554 webserver.cc:460] Webserver started at http://127.9.91.126:43255/ using document root <none> and password file <none>
I20260809 19:22:28.218139 26554 fs_manager.cc:359] Metadata directory not provided
I20260809 19:22:28.218192 26554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:22:28.218323 26554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:22:28 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303348, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:22:28.263027 26554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "4d8feb43de864114ba65fdff954c427d"
format_stamp: "Formatted at 2026-08-09 19:22:28 on dist-test-slave-1vmg"
server_key: "f4b3eb31e07a2ef46e00a462096ab4f4"
server_key_iv: "8e5e4d5c51a907e8b8098ff251c6f4fb"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:28.263478 26554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "4d8feb43de864114ba65fdff954c427d"
format_stamp: "Formatted at 2026-08-09 19:22:28 on dist-test-slave-1vmg"
server_key: "f4b3eb31e07a2ef46e00a462096ab4f4"
server_key_iv: "8e5e4d5c51a907e8b8098ff251c6f4fb"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:28.265291 26554 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 19:22:28.278807 26581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:28.279050 26554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:22:28.279156 26554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "4d8feb43de864114ba65fdff954c427d"
format_stamp: "Formatted at 2026-08-09 19:22:28 on dist-test-slave-1vmg"
server_key: "f4b3eb31e07a2ef46e00a462096ab4f4"
server_key_iv: "8e5e4d5c51a907e8b8098ff251c6f4fb"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:28.279235 26554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:22:28.297821 26554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:22:28.299671 26554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:22:28.299816 26554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:22:28.304028 26554 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:35549
I20260809 19:22:28.304078 26664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:35549 every 8 connection(s)
I20260809 19:22:28.304400 26554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:22:28.305151 26665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:22:28.308264 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 26554
I20260809 19:22:28.308362 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:22:28.308470 26665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d: Bootstrap starting.
I20260809 19:22:28.308526 9581 external_mini_cluster.cc:1428] Setting key de99c11bca5004de442a8e4823409ede
I20260809 19:22:28.309166 26665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d: Neither blocks nor log segments found. Creating new log.
I20260809 19:22:28.309500 26665 log.cc:824] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:22:28.311022 26665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d: No bootstrap required, opened a new log
I20260809 19:22:28.313511 26665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8feb43de864114ba65fdff954c427d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35549 } }
I20260809 19:22:28.313683 26665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:22:28.313717 26665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8feb43de864114ba65fdff954c427d, State: Initialized, Role: FOLLOWER
I20260809 19:22:28.313859 26665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8feb43de864114ba65fdff954c427d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35549 } }
I20260809 19:22:28.313941 26665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:22:28.313969 26665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:22:28.314023 26665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 0 FOLLOWER]: Advancing to term 1
Aug 09 19:22:28 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:22:28.314656 26665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8feb43de864114ba65fdff954c427d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35549 } }
I20260809 19:22:28.314762 26665 leader_election.cc:302] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d8feb43de864114ba65fdff954c427d; no voters:
I20260809 19:22:28.314940 26665 leader_election.cc:288] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:22:28.315284 26665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:22:28.315351 26676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:22:28.315513 26676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [term 1 LEADER]: Becoming Leader. State: Replica: 4d8feb43de864114ba65fdff954c427d, State: Running, Role: LEADER
I20260809 19:22:28.315994 26676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8feb43de864114ba65fdff954c427d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35549 } }
I20260809 19:22:28.317598 26677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d8feb43de864114ba65fdff954c427d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8feb43de864114ba65fdff954c427d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35549 } } }
I20260809 19:22:28.317696 26677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [sys.catalog]: This master's current role is: LEADER
I20260809 19:22:28.317941 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d8feb43de864114ba65fdff954c427d. Latest consensus state: current_term: 1 leader_uuid: "4d8feb43de864114ba65fdff954c427d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8feb43de864114ba65fdff954c427d" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35549 } } }
I20260809 19:22:28.318011 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d [sys.catalog]: This master's current role is: LEADER
I20260809 19:22:28.318346 26665 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:22:28.322130 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:28.308981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35208 (local address 127.9.91.126:35549)
0809 19:22:28.309229 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:28.309238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:28.309271 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:22:28.309712 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:28.309722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:28.309789 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:28.309944 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:22:28.310650 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:28.311594 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:28.312319 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:28.312738 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:28.312833 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:28.315070 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:22:28.315075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:22:28.315075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:22:28.315088 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:22:28.316821 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:28.317189 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:28.317192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:28.317192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:28.317277 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:28.318736 (+ 1459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:28.318736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:28.318737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:28.320740 (+ 2003us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:22:28.321326 (+ 586us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:28.321448 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 19:22:28.321797 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":81,"threads_started":1}
I20260809 19:22:28.322445 26699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:22:28.323781 26699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:22:28.325659 26699 catalog_manager.cc:1347] Generated new cluster ID: 87235097e1724a15bde32117ce24b9b6
I20260809 19:22:28.325721 26699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:22:28.348587 26699 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:22:28.349275 26699 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:22:28.357268 26699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d: Generated new TSK 0
I20260809 19:22:28.357420 26699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:22:29.047472 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:50183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35549
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:22:29.164628 26733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:22:29.164880 26733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:22:29.164919 26733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:22:29.166515 26733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:22:29.166574 26733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:22:29.166617 26733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:22:29.168767 26733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.91.84:50183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:22:29.169023 26733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:22:29.169283 26733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:22:29.174373 26743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:22:29.174325 26740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:22:29.174335 26738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:22:29.174656 26733 server_base.cc:1034] running on GCE node
I20260809 19:22:29.174870 26733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:22:29.175107 26733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:22:29.176236 26733 hybrid_clock.cc:648] HybridClock initialized: now 1786303349176224 us; error 31 us; skew 500 ppm
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:22:29.178606 26733 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:22:29.179060 26733 webserver.cc:460] Webserver started at http://127.9.91.65:39871/ using document root <none> and password file <none>
I20260809 19:22:29.179273 26733 fs_manager.cc:359] Metadata directory not provided
I20260809 19:22:29.179315 26733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:22:29.179419 26733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:22:29.200554 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "ffb3f84febbd40aca9f2d92de5e62f02"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "adb3606d7b0c1a33ecb0c3fd4feeed5c"
server_key_iv: "4fb752a71a78c6759552c4076fb5b12f"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.201072 26733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "ffb3f84febbd40aca9f2d92de5e62f02"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "adb3606d7b0c1a33ecb0c3fd4feeed5c"
server_key_iv: "4fb752a71a78c6759552c4076fb5b12f"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.204002 26733 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 19:22:29.219689 26756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.224550 26733 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260809 19:22:29.224653 26733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "ffb3f84febbd40aca9f2d92de5e62f02"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "adb3606d7b0c1a33ecb0c3fd4feeed5c"
server_key_iv: "4fb752a71a78c6759552c4076fb5b12f"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.224735 26733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:22:29.254786 26733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:22:29.257186 26733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:22:29.257320 26733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:22:29.257542 26733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:22:29.257918 26733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:22:29.257999 26733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.258054 26733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:22:29.258095 26733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.264830 26733 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:34947
I20260809 19:22:29.265301 26733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:22:29.265905 26924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:34947 every 8 connection(s)
I20260809 19:22:29.275100 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 26733
I20260809 19:22:29.275218 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:22:29.275408 9581 external_mini_cluster.cc:1428] Setting key 87994a4751263019c69ae9d765c4c776
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:22:29.282027 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.266439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35549 (local address 127.9.91.65:48589)
0809 19:22:29.271003 (+ 4564us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:29.271059 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:29.271689 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:29.272141 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:29.272146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:29.272381 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:22:29.272658 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.272665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.273683 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.273685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.274979 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.274982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.275530 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.275531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.275693 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.276606 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:22:29.276616 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:22:29.278255 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:22:29.280361 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:22:29.280367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:22:29.280370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:22:29.280646 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:22:29.280884 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:22:29.280885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:22:29.280885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:22:29.281018 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:22:29.281375 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:22:29.281381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:22:29.281547 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 19:22:29.281652 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 19:22:29.281808 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4516,"thread_start_us":87,"threads_started":1}
I20260809 19:22:29.282529 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.266739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:48589 (local address 127.9.91.126:35549)
0809 19:22:29.271879 (+ 5140us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:29.271881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:29.271889 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:22:29.271917 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:29.271917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:29.271930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:29.271997 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:22:29.272776 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.273567 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.275087 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.275393 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.275459 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.278361 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:22:29.278363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:22:29.278363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:22:29.278371 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:22:29.280272 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.280745 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.280747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.280747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.280816 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.281086 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.281087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.281088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.281220 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:22:29.281345 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:29.282344 (+ 999us) server_negotiation.cc:299] Negotiation successful
0809 19:22:29.282433 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5103,"thread_start_us":82,"threads_started":1}
I20260809 19:22:29.283361 26926 heartbeater.cc:347] Connected to a master server at 127.9.91.126:35549
I20260809 19:22:29.283445 26926 heartbeater.cc:464] Registering TS with master...
I20260809 19:22:29.283609 26926 heartbeater.cc:511] Master 127.9.91.126:35549 requested a full tablet report, sending...
I20260809 19:22:29.284188 26610 ts_manager.cc:194] Registered new tserver with Master: ffb3f84febbd40aca9f2d92de5e62f02 (127.9.91.65:34947)
I20260809 19:22:29.285066 26610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:48589
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:22:29.335843 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:50183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35549
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 19:22:29.460765 26938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:22:29.461084 26938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:22:29.461154 26938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:22:29.462846 26938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:22:29.462958 26938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:22:29.463038 26938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:22:29.465144 26938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.91.84:50183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:22:29.465378 26938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:22:29.465644 26938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:22:29.473775 26951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:22:29.474069 26947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:22:29.474248 26946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:22:29.477895 26938 server_base.cc:1034] running on GCE node
I20260809 19:22:29.478108 26938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:22:29.478355 26938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:22:29.493846 26938 hybrid_clock.cc:648] HybridClock initialized: now 1786303349493810 us; error 55 us; skew 500 ppm
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:22:29.497335 26938 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:22:29.497838 26938 webserver.cc:460] Webserver started at http://127.9.91.66:42667/ using document root <none> and password file <none>
I20260809 19:22:29.498102 26938 fs_manager.cc:359] Metadata directory not provided
I20260809 19:22:29.498150 26938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:22:29.498270 26938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:22:29.520696 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "fe864a419719785298b18dd26c3737ae"
server_key_iv: "c438fc25af7cacb4491bec0ae51ef864"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.521178 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "fe864a419719785298b18dd26c3737ae"
server_key_iv: "c438fc25af7cacb4491bec0ae51ef864"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.523097 26938 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 19:22:29.534576 26961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.534767 26938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:22:29.534847 26938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "fe864a419719785298b18dd26c3737ae"
server_key_iv: "c438fc25af7cacb4491bec0ae51ef864"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.534937 26938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:22:29.549542 26938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:22:29.551877 26938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:22:29.552058 26938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:22:29.552291 26938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:22:29.552659 26938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:22:29.552742 26938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.552811 26938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:22:29.552866 26938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.561381 26938 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:35105
I20260809 19:22:29.561861 26938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:22:29.562414 27138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:35105 every 8 connection(s)
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:22:29.570076 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 26938
I20260809 19:22:29.570192 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:22:29.570389 9581 external_mini_cluster.cc:1428] Setting key d4ac606bbd335278b29ba7f8461d1d84
I20260809 19:22:29.573859 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.563069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:41007 (local address 127.9.91.126:35549)
0809 19:22:29.563131 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:29.563134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:29.563847 (+ 713us) server_negotiation.cc:407] Connection header received
0809 19:22:29.564621 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:29.564622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:29.564635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:29.564666 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:22:29.565474 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.566261 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.567755 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.568065 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.568111 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.570633 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:22:29.570635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:22:29.570635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:22:29.570642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:22:29.572330 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.572792 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.572793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.572793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.572852 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.573117 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.573118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.573118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.573246 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:22:29.573285 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:29.573677 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 19:22:29.573766 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:22:29.574594 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.563194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35549 (local address 127.9.91.66:41007)
0809 19:22:29.563720 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:29.563764 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:29.564420 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:29.564852 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:29.564856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:29.565088 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:22:29.565361 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.565367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.566379 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.566379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.567654 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.567657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.568203 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.568204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.568343 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.568849 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:22:29.568859 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:22:29.570533 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:22:29.572417 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:22:29.572423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:22:29.572425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:22:29.572705 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:22:29.572919 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:22:29.572920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:22:29.572920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:22:29.573045 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:22:29.573328 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:22:29.573334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:22:29.573500 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 19:22:29.574224 (+ 724us) client_negotiation.cc:240] Negotiation successful
0809 19:22:29.574380 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":87,"threads_started":1}
I20260809 19:22:29.575225 27139 heartbeater.cc:347] Connected to a master server at 127.9.91.126:35549
I20260809 19:22:29.575307 27139 heartbeater.cc:464] Registering TS with master...
I20260809 19:22:29.575472 27139 heartbeater.cc:511] Master 127.9.91.126:35549 requested a full tablet report, sending...
I20260809 19:22:29.575865 26610 ts_manager.cc:194] Registered new tserver with Master: 110c6b309ea04d5a9874e5fb1c1e9e7f (127.9.91.66:35105)
I20260809 19:22:29.576335 26610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:41007
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:22:29.621711 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:50183/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35549
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:22:29.775220 27150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:22:29.775553 27150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:22:29.775652 27150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:22:29.777295 27150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:22:29.777356 27150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:22:29.777395 27150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:22:29.779531 27150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.91.84:50183/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:35549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:22:29.779783 27150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:22:29.780084 27150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:22:29.786368 27158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:22:29.786504 27164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:22:29.786862 27161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:22:29.787209 27150 server_base.cc:1034] running on GCE node
I20260809 19:22:29.787405 27150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:22:29.787663 27150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:22:29.788831 27150 hybrid_clock.cc:648] HybridClock initialized: now 1786303349788809 us; error 34 us; skew 500 ppm
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:22:29.792068 27150 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:22:29.792587 27150 webserver.cc:460] Webserver started at http://127.9.91.67:44297/ using document root <none> and password file <none>
I20260809 19:22:29.792833 27150 fs_manager.cc:359] Metadata directory not provided
I20260809 19:22:29.792901 27150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:22:29.793035 27150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:22:29.807715 27150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "ff338fd10b2f417db026998ace6cbed4"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "34da98d7897c62e271202df58bbbf80a"
server_key_iv: "3d57a376a01a0317045f08194aafcc8b"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.808203 27150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "ff338fd10b2f417db026998ace6cbed4"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "34da98d7897c62e271202df58bbbf80a"
server_key_iv: "3d57a376a01a0317045f08194aafcc8b"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.815265 27150 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260809 19:22:29.828385 27176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.837802 27150 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260809 19:22:29.837918 27150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "ff338fd10b2f417db026998ace6cbed4"
format_stamp: "Formatted at 2026-08-09 19:22:29 on dist-test-slave-1vmg"
server_key: "34da98d7897c62e271202df58bbbf80a"
server_key_iv: "3d57a376a01a0317045f08194aafcc8b"
server_key_version: "kuduclusterkey@0"
I20260809 19:22:29.837997 27150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:22:29.849035 27150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:22:29.851348 27150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:22:29.851491 27150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:22:29.851724 27150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:22:29.852079 27150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:22:29.852129 27150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.852160 27150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:22:29.852182 27150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:22:29.858639 27150 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:34129
I20260809 19:22:29.859093 27150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:22:29.865449 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 27150
I20260809 19:22:29.865581 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:22:29.865662 27351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:34129 every 8 connection(s)
I20260809 19:22:29.865789 9581 external_mini_cluster.cc:1428] Setting key 1ef0b2fda35648c85b0a07dfa191d220
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:22:29.881352 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.867128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:33261 (local address 127.9.91.126:35549)
0809 19:22:29.867170 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:29.867172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:29.871305 (+ 4133us) server_negotiation.cc:407] Connection header received
0809 19:22:29.871947 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:29.871948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:29.871961 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:29.872009 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:22:29.872773 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.873554 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.875026 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.875320 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.875358 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.878240 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:22:29.878241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:22:29.878242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:22:29.878249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:22:29.879881 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.880340 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.880341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.880342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.880403 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.880643 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.880644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.880644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.880766 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:22:29.880805 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:29.881183 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 19:22:29.881263 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:22:29.881796 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.867136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35549 (local address 127.9.91.67:33261)
0809 19:22:29.871146 (+ 4010us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:29.871201 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:29.871825 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:29.872151 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:29.872156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:29.872389 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:22:29.872666 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.872671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.873666 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.873667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.874902 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.874905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.875442 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.875442 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.875594 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.876113 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:22:29.876123 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:22:29.878136 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:22:29.879951 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:22:29.879958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:22:29.879960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:22:29.880248 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:22:29.880464 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:22:29.880465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:22:29.880465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:22:29.880577 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:22:29.880848 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:22:29.880854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:22:29.881026 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 19:22:29.881438 (+ 412us) client_negotiation.cc:240] Negotiation successful
0809 19:22:29.881584 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3959,"thread_start_us":81,"threads_started":1}
I20260809 19:22:29.882843 27352 heartbeater.cc:347] Connected to a master server at 127.9.91.126:35549
I20260809 19:22:29.882953 27352 heartbeater.cc:464] Registering TS with master...
I20260809 19:22:29.883141 27352 heartbeater.cc:511] Master 127.9.91.126:35549 requested a full tablet report, sending...
I20260809 19:22:29.883456 26610 ts_manager.cc:194] Registered new tserver with Master: ff338fd10b2f417db026998ace6cbed4 (127.9.91.67:34129)
I20260809 19:22:29.883896 26610 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:33261
I20260809 19:22:29.888576 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 19:22:29 dist-test-slave-1vmg krb5kdc[25867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303349, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:22:29.919131 26929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.899174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35220 (local address 127.9.91.126:35549)
0809 19:22:29.899217 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:29.899218 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:29.908206 (+ 8988us) server_negotiation.cc:407] Connection header received
0809 19:22:29.908306 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:29.908307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:29.908318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:29.908345 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:22:29.909084 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.909816 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.910931 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.911228 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.911271 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.912729 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:22:29.912731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:22:29.912731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:22:29.912738 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:22:29.914345 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.914748 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.914749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.914750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.914809 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:22:29.915052 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:22:29.915053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:22:29.915054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:22:29.915182 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:22:29.915257 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:29.918933 (+ 3676us) server_negotiation.cc:299] Negotiation successful
0809 19:22:29.919030 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:22:29.925909 26610 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:22:29.928228 26610 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:22:29.939401 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.934345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:34947 (local address 127.0.0.1:33874)
0809 19:22:29.934733 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:29.934777 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:29.934841 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:29.935277 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:29.935279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:29.935281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:22:29.935348 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.935349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.936594 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.936595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.937821 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.937824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.939108 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.939109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.939184 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.939199 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 19:22:29.939241 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 19:22:29.939333 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":96,"threads_started":1}
I20260809 19:22:29.943542 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.934604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:34129 (local address 127.0.0.1:53582)
0809 19:22:29.939674 (+ 5070us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:29.939684 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:29.939725 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:29.940049 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:29.940050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:29.940060 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:22:29.940088 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.940089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.941246 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.941247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.942274 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.942277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.943284 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.943285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.943402 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.943406 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:22:29.943452 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:22:29.943483 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5061,"thread_start_us":59,"threads_started":1}
I20260809 19:22:29.943903 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.934809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.9.91.67:34129)
0809 19:22:29.935028 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:29.935033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:29.939834 (+ 4801us) server_negotiation.cc:407] Connection header received
0809 19:22:29.939879 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:29.939881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:29.939897 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:29.939961 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:22:29.940165 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.941158 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.942379 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.943120 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.943182 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.943250 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:29.943790 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 19:22:29.943830 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
I20260809 19:22:29.944329 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.934619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.9.91.65:34947)
0809 19:22:29.935107 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:29.935111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:29.935121 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:22:29.935155 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:29.935157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:29.935173 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:29.935412 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:22:29.935471 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.936492 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.937921 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.939009 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.944130 (+ 5121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.944198 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:29.944223 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 19:22:29.944265 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":60,"threads_started":1}
I20260809 19:22:29.945155 26826 tablet_service.cc:1467] Processing CreateTablet for tablet 7cc80f7751c44142b38aa2cfefb6e1b5 (DEFAULT_TABLE table=test-table [id=9d0bd0379d794ea192615e1588c8af94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:22:29.945318 27246 tablet_service.cc:1467] Processing CreateTablet for tablet 7cc80f7751c44142b38aa2cfefb6e1b5 (DEFAULT_TABLE table=test-table [id=9d0bd0379d794ea192615e1588c8af94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:22:29.945544 26826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cc80f7751c44142b38aa2cfefb6e1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:22:29.945681 27246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cc80f7751c44142b38aa2cfefb6e1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:22:29.948563 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.934347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:35105 (local address 127.0.0.1:42112)
0809 19:22:29.938250 (+ 3903us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:29.938259 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:29.938307 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:29.944906 (+ 6599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:29.944907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:29.944909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:22:29.944945 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.944947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.946363 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.946364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.947408 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:29.947412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.948364 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.948365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:29.948426 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.948430 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:22:29.948460 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 19:22:29.948493 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3886,"thread_start_us":107,"threads_started":1}
I20260809 19:22:29.949033 27368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:29.934459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.9.91.66:35105)
0809 19:22:29.944733 (+ 10274us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:29.944738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:29.944749 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:22:29.944789 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:29.944791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:29.944806 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:29.945006 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:22:29.945096 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.946264 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.947537 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:29.948257 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:29.948810 (+ 553us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:29.948895 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:29.948921 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 19:22:29.948958 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10122,"thread_start_us":90,"threads_started":1}
I20260809 19:22:29.949766 27036 tablet_service.cc:1467] Processing CreateTablet for tablet 7cc80f7751c44142b38aa2cfefb6e1b5 (DEFAULT_TABLE table=test-table [id=9d0bd0379d794ea192615e1588c8af94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:22:29.950105 27036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cc80f7751c44142b38aa2cfefb6e1b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:22:29.954107 27375 tablet_bootstrap.cc:492] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Bootstrap starting.
I20260809 19:22:29.956687 27375 tablet_bootstrap.cc:654] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Neither blocks nor log segments found. Creating new log.
I20260809 19:22:29.957026 27375 log.cc:824] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Log is configured to *not* fsync() on all Append() calls
I20260809 19:22:29.963467 27374 tablet_bootstrap.cc:492] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02: Bootstrap starting.
I20260809 19:22:29.964296 27374 tablet_bootstrap.cc:654] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02: Neither blocks nor log segments found. Creating new log.
I20260809 19:22:29.964625 27374 log.cc:824] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02: Log is configured to *not* fsync() on all Append() calls
I20260809 19:22:29.965477 27375 tablet_bootstrap.cc:492] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: No bootstrap required, opened a new log
I20260809 19:22:29.965557 27375 ts_tablet_manager.cc:1397] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260809 19:22:29.967546 27375 raft_consensus.cc:348] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:29.967729 27375 raft_consensus.cc:374] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:22:29.967770 27375 raft_consensus.cc:729] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff338fd10b2f417db026998ace6cbed4, State: Initialized, Role: FOLLOWER
I20260809 19:22:29.968025 27376 tablet_bootstrap.cc:492] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: Bootstrap starting.
I20260809 19:22:29.969348 27376 tablet_bootstrap.cc:654] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: Neither blocks nor log segments found. Creating new log.
I20260809 19:22:29.969650 27376 log.cc:824] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:22:29.969930 27375 consensus_queue.cc:260] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:29.970247 27375 ts_tablet_manager.cc:1428] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260809 19:22:29.970376 27352 heartbeater.cc:503] Master 127.9.91.126:35549 was elected leader, sending a full tablet report...
I20260809 19:22:29.971832 27376 tablet_bootstrap.cc:492] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: No bootstrap required, opened a new log
I20260809 19:22:29.971907 27376 ts_tablet_manager.cc:1397] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 19:22:29.973763 27376 raft_consensus.cc:348] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:29.973932 27376 raft_consensus.cc:374] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:22:29.973981 27376 raft_consensus.cc:729] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110c6b309ea04d5a9874e5fb1c1e9e7f, State: Initialized, Role: FOLLOWER
I20260809 19:22:29.974133 27376 consensus_queue.cc:260] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:29.974403 27376 ts_tablet_manager.cc:1428] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:22:29.974524 27139 heartbeater.cc:503] Master 127.9.91.126:35549 was elected leader, sending a full tablet report...
I20260809 19:22:29.976818 27374 tablet_bootstrap.cc:492] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02: No bootstrap required, opened a new log
I20260809 19:22:29.976902 27374 ts_tablet_manager.cc:1397] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260809 19:22:29.979575 27374 raft_consensus.cc:348] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:29.979765 27374 raft_consensus.cc:374] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:22:29.979812 27374 raft_consensus.cc:729] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffb3f84febbd40aca9f2d92de5e62f02, State: Initialized, Role: FOLLOWER
I20260809 19:22:29.979972 27374 consensus_queue.cc:260] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:29.980278 27374 ts_tablet_manager.cc:1428] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 19:22:29.980415 26926 heartbeater.cc:503] Master 127.9.91.126:35549 was elected leader, sending a full tablet report...
W20260809 19:22:30.023888 26927 tablet.cc:2369] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:22:30.063954 27140 tablet.cc:2369] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:22:30.107486 27389 raft_consensus.cc:482] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:22:30.107646 27389 raft_consensus.cc:504] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:30.108032 27389 leader_election.cc:288] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 110c6b309ea04d5a9874e5fb1c1e9e7f (127.9.91.66:35105), ff338fd10b2f417db026998ace6cbed4 (127.9.91.67:34129)
I20260809 19:22:30.112627 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:30.108863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:55831 (local address 127.9.91.67:34129)
0809 19:22:30.108919 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:30.108921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:30.109170 (+ 249us) server_negotiation.cc:407] Connection header received
0809 19:22:30.109253 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:30.109254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:30.109265 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:30.109292 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:22:30.109494 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.110220 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.111691 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.112265 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.112310 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:30.112319 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:30.112522 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 19:22:30.112557 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:22:30.113036 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:30.108488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:34129 (local address 127.9.91.65:55831)
0809 19:22:30.109124 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:30.109132 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:30.109188 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:30.109351 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:30.109352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:30.109353 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:22:30.109429 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:30.109430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.110336 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.110337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:30.111593 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:30.111596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.112376 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.112377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:30.112448 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:30.112452 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:22:30.112901 (+ 449us) client_negotiation.cc:240] Negotiation successful
0809 19:22:30.112941 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":73,"threads_started":1}
I20260809 19:22:30.113476 27282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc80f7751c44142b38aa2cfefb6e1b5" candidate_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff338fd10b2f417db026998ace6cbed4" is_pre_election: true
I20260809 19:22:30.113651 27282 raft_consensus.cc:2454] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffb3f84febbd40aca9f2d92de5e62f02 in term 0.
I20260809 19:22:30.113880 26758 leader_election.cc:302] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ff338fd10b2f417db026998ace6cbed4, ffb3f84febbd40aca9f2d92de5e62f02; no voters:
I20260809 19:22:30.114010 27389 raft_consensus.cc:2781] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:22:30.114060 27389 raft_consensus.cc:482] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:22:30.114085 27389 raft_consensus.cc:3037] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:22:30.115005 27389 raft_consensus.cc:504] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:30.115119 27389 leader_election.cc:288] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [CANDIDATE]: Term 1 election: Requested vote from peers 110c6b309ea04d5a9874e5fb1c1e9e7f (127.9.91.66:35105), ff338fd10b2f417db026998ace6cbed4 (127.9.91.67:34129)
I20260809 19:22:30.115306 27282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc80f7751c44142b38aa2cfefb6e1b5" candidate_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff338fd10b2f417db026998ace6cbed4"
I20260809 19:22:30.115377 27282 raft_consensus.cc:3037] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:22:30.115772 27368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:30.108805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35571 (local address 127.9.91.66:35105)
0809 19:22:30.108869 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:30.108871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:30.109009 (+ 138us) server_negotiation.cc:407] Connection header received
0809 19:22:30.109095 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:30.109096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:30.109109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:30.109181 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:22:30.109416 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.110142 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.114757 (+ 4615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.115285 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.115327 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:30.115338 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:30.115667 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 19:22:30.115711 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:22:30.116149 27392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:30.108312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:35105 (local address 127.9.91.65:35571)
0809 19:22:30.108943 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:30.108955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:22:30.109034 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:22:30.109241 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:22:30.109242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:22:30.109245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:22:30.109340 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:30.109342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.110238 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.110239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:30.114653 (+ 4414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:22:30.114656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.115501 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.115502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:22:30.115577 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:30.115580 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:22:30.115997 (+ 417us) client_negotiation.cc:240] Negotiation successful
0809 19:22:30.116051 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":2951,"thread_start_us":85,"threads_started":1}
I20260809 19:22:30.116333 27282 raft_consensus.cc:2454] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffb3f84febbd40aca9f2d92de5e62f02 in term 1.
I20260809 19:22:30.116667 26758 leader_election.cc:302] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ff338fd10b2f417db026998ace6cbed4, ffb3f84febbd40aca9f2d92de5e62f02; no voters:
I20260809 19:22:30.116629 27060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc80f7751c44142b38aa2cfefb6e1b5" candidate_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" is_pre_election: true
I20260809 19:22:30.116770 27060 raft_consensus.cc:2454] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffb3f84febbd40aca9f2d92de5e62f02 in term 0.
W20260809 19:22:30.116961 27353 tablet.cc:2369] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:22:30.116633 27059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cc80f7751c44142b38aa2cfefb6e1b5" candidate_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f"
I20260809 19:22:30.117316 27059 raft_consensus.cc:3037] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:22:30.118098 27059 raft_consensus.cc:2454] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffb3f84febbd40aca9f2d92de5e62f02 in term 1.
I20260809 19:22:30.118341 27389 raft_consensus.cc:2781] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:22:30.118489 27389 raft_consensus.cc:686] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [term 1 LEADER]: Becoming Leader. State: Replica: ffb3f84febbd40aca9f2d92de5e62f02, State: Running, Role: LEADER
I20260809 19:22:30.118592 27389 consensus_queue.cc:237] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } }
I20260809 19:22:30.119390 26610 catalog_manager.cc:5495] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffb3f84febbd40aca9f2d92de5e62f02 (127.9.91.65). New cstate: current_term: 1 leader_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffb3f84febbd40aca9f2d92de5e62f02" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 34947 } health_report { overall_health: HEALTHY } } }
I20260809 19:22:30.170549 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:30.167149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33886 (local address 127.9.91.65:34947)
0809 19:22:30.167219 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 19:22:30.167222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:22:30.167230 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:22:30.167270 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:22:30.167271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:22:30.167295 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:22:30.167366 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:22:30.167560 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.168293 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.169406 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:22:30.169719 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:22:30.169783 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:22:30.169981 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:22:30.170117 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:22:30.170191 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:22:30.170417 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 19:22:30.170438 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 19:22:30.177331 27282 raft_consensus.cc:1261] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 1 FOLLOWER]: Refusing update from remote peer ffb3f84febbd40aca9f2d92de5e62f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:22:30.177805 27394 consensus_queue.cc:1035] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff338fd10b2f417db026998ace6cbed4" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:22:30.180200 27059 raft_consensus.cc:1261] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 1 FOLLOWER]: Refusing update from remote peer ffb3f84febbd40aca9f2d92de5e62f02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:22:30.180539 27394 consensus_queue.cc:1035] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ffb3f84febbd40aca9f2d92de5e62f02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "110c6b309ea04d5a9874e5fb1c1e9e7f" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 35105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:22:30.184346 27400 mvcc.cc:205] Tried to move back new op lower bound from 7316698522304057344 to 7316698522086187008. Current Snapshot: MvccSnapshot[applied={T|T < 7316698522304057344}]
I20260809 19:22:30.185509 27405 mvcc.cc:205] Tried to move back new op lower bound from 7316698522304057344 to 7316698522086187008. Current Snapshot: MvccSnapshot[applied={T|T < 7316698522304057344}]
I20260809 19:22:30.187405 27403 mvcc.cc:205] Tried to move back new op lower bound from 7316698522304057344 to 7316698522086187008. Current Snapshot: MvccSnapshot[applied={T|T < 7316698522304057344}]
I20260809 19:22:30.189790 26610 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:22:30.189879 26610 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:22:30.190676 26610 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d8feb43de864114ba65fdff954c427d: Sending DeleteTablet for 3 replicas of tablet 7cc80f7751c44142b38aa2cfefb6e1b5
I20260809 19:22:30.191108 27036 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cc80f7751c44142b38aa2cfefb6e1b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:22:30 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:42112
I20260809 19:22:30.191326 27246 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cc80f7751c44142b38aa2cfefb6e1b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:22:30 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53582
I20260809 19:22:30.191800 26826 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cc80f7751c44142b38aa2cfefb6e1b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:22:30 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:33874
I20260809 19:22:30.192061 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 26733
I20260809 19:22:30.204309 27414 tablet_replica.cc:330] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: stopping tablet replica
I20260809 19:22:30.204464 27415 tablet_replica.cc:330] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: stopping tablet replica
I20260809 19:22:30.204638 27415 raft_consensus.cc:2229] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:22:30.204797 27414 raft_consensus.cc:2229] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:22:30.204950 27414 raft_consensus.cc:2258] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:22:30.205083 27415 raft_consensus.cc:2258] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:22:30.207197 27415 ts_tablet_manager.cc:1905] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:22:30.208910 27415 ts_tablet_manager.cc:1918] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:22:30.209072 27415 log.cc:1196] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/7cc80f7751c44142b38aa2cfefb6e1b5
I20260809 19:22:30.209718 27415 ts_tablet_manager.cc:1939] T 7cc80f7751c44142b38aa2cfefb6e1b5 P ff338fd10b2f417db026998ace6cbed4: Deleting consensus metadata
I20260809 19:22:30.210206 26588 catalog_manager.cc:4841] TS ff338fd10b2f417db026998ace6cbed4 (127.9.91.67:34129): tablet 7cc80f7751c44142b38aa2cfefb6e1b5 (table test-table [id=9d0bd0379d794ea192615e1588c8af94]) successfully deleted
I20260809 19:22:30.215202 27414 ts_tablet_manager.cc:1905] T 7cc80f7751c44142b38aa2cfefb6e1b5 P 110c6b309ea04d5a9874e5fb1c1e9e7f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 19:22:30.215461 26587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:34947 (error 108)
I20260809 19:22:30.216192 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 26938
I20260809 19:22:30.218293 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:30.218179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:34947 (local address 127.0.0.1:33896)
0809 19:22:30.218226 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:30.218257 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:34947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 19:22:30.218490 26587 catalog_manager.cc:4584] TS ffb3f84febbd40aca9f2d92de5e62f02 (127.9.91.65:34947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cc80f7751c44142b38aa2cfefb6e1b5: Network error: Client connection negotiation failed: client connection to 127.9.91.65:34947: connect: Connection refused (error 111)
I20260809 19:22:30.223592 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 27150
I20260809 19:22:30.225473 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:22:30.225381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:35105 (local address 127.0.0.1:42118)
0809 19:22:30.225422 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:22:30.225444 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.66:35105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:22:30.231247 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 26554
I20260809 19:22:30.244534 9581 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 19:22:30.587827 9581 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 19:22:30.587929 9581 mini_ranger.cc:66] Stopping Ranger...
I20260809 19:22:30.931283 9581 mini_ranger.cc:68] Stopped Ranger
2026-08-09 19:22:30.931 UTC [25912] LOG: received smart shutdown request
2026-08-09 19:22:30.934 UTC [25912] LOG: background worker "logical replication launcher" (PID 25925) exited with exit code 1
2026-08-09 19:22:30.935 UTC [25917] LOG: shutting down
2026-08-09 19:22:31.062 UTC [25912] LOG: database system is shut down
2026-08-09T19:22:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45394 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:22:31 dist-test-slave-1vmg krb5kdc[27430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:22:31 dist-test-slave-1vmg krb5kdc[27430](info): set up 2 sockets
Aug 09 19:22:31 dist-test-slave-1vmg krb5kdc[27430](info): commencing operation
krb5kdc: starting...
W20260809 19:22:32.288059 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:22:32 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:22:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:22:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/rangeradmin_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/rangeradmin_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/rangerlookup_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/rangerlookup_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
I20260809 19:22:32.429327 9581 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskt8E36c/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/postgres -l logfile start
2026-08-09 19:22:34.520 UTC [27478] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 19:22:34.520 UTC [27478] LOG: listening on IPv4 address "127.9.91.84", port 56131
2026-08-09 19:22:34.523 UTC [27478] LOG: listening on Unix socket "/tmp/.s.PGSQL.56131"
2026-08-09 19:22:34.542 UTC [27484] LOG: database system was shut down at 2026-08-09 19:22:33 UTC
2026-08-09 19:22:34.546 UTC [27478] LOG: database system is ready to accept connections
I20260809 19:22:35.585413 9581 mini_postgres.cc:96] Postgres bound to 56131
2026-08-09 19:22:35.592 UTC [27496] FATAL: database "slave" does not exist
127.9.91.84:56131 - accepting connections
I20260809 19:22:35.592657 9581 mini_ranger.cc:161] Starting Ranger...
I20260809 19:22:35.606527 9581 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 19:22:35.976606 9581 mini_ranger.cc:87] Created ranger Postgres database
I20260809 19:22:35.976691 9581 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-admin
2026-08-09 19:22:36,237 [I] DB FLAVOR :POSTGRES
2026-08-09 19:22:36,237 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:22:36,237 [I] Checking connection..
2026-08-09 19:22:36,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 19:22:36,477 [I] Checking connection passed.
2026-08-09 19:22:36,477 [I] --------- Verifying version history table ---------
2026-08-09 19:22:36,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:22:36,734 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:22:36,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:22:36,988 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 19:22:36,989 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 19:22:36,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 19:22:37.236 UTC [27604] WARNING: there is no transaction in progress
2026-08-09 19:22:37,242 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 19:22:37,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 19:22:37,492 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 19:22:37,493 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:22:37,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 19:22:37,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 19:22:37,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1vmg.c.gcp-upstream.internal','N') ;"
2026-08-09 19:22:38,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:22:38,453 [I] Table x_portal_user does not exist in database ranger
2026-08-09 19:22:38,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:22:38,716 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 19:22:38,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:22:38,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:22:39,222 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 19:22:39,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 19:22:40.090 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.104 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.118 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.131 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.145 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.159 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.206 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.211 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.223 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.232 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.240 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.251 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.259 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.264 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.270 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40.275 UTC [27770] WARNING: there is no transaction in progress
2026-08-09 19:22:40,792 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 19:22:40,792 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 19:22:41,052 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 19:22:41,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 19:22:41,285 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 19:22:41,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 19:22:41,523 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 19:22:41,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 19:22:41,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 19:22:42,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1vmg.c.gcp-upstream.internal';"
2026-08-09 19:22:42,241 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 19:22:42,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 19:22:42,478 [I] DB_PATCHES have already been applied
I20260809 19:22:42.484315 9581 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-admin:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskt8E36c/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 19:22:42.484373 9581 mini_ranger.cc:193] Using host: 127.9.91.84
I20260809 19:22:42.487953 9581 mini_ranger.cc:227] Ranger admin URL: http://127.9.91.84:42259
Aug 09, 2026 7:22:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:22:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:22:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:22:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:22:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:22:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:22:43 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 19:22:43.675799 9581 mini_ranger.cc:160] Time spent starting Ranger: real 8.083s user 0.001s sys 0.008s
Aug 09, 2026 7:23:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:23:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 7:23:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:23:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:23:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 19:23:03.580726 9581 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.91.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.91.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/rangerkms_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.91.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/rangerkms_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.91.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.91.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.91.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/HTTP_127.9.91.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 19:23:03 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303383, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 19:23:03.652570 9581 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 19:23:03.660614 9581 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 19:23:04.339639 9581 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 19:23:04.339715 9581 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms
I20260809 19:23:04.339732 9581 mini_ranger_kms.cc:227] Using postgres at 127.9.91.84:56131
2026-08-09 19:23:04,500 [I] DB FLAVOR :POSTGRES
2026-08-09 19:23:04,500 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 19:23:04,501 [I] Checking connection
2026-08-09 19:23:04,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 19:23:04,737 [I] connection success
2026-08-09 19:23:04,737 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 19:23:04,737 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 19:23:04,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 19:23:04,960 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 19:23:04,960 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 19:23:04,960 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 19:23:04,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.91.84:56131/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 19:23:05,223 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 19:23:05.689872 9581 mini_ranger_kms.cc:310] Created kms service
I20260809 19:23:05.861157 9581 mini_ranger_kms.cc:326] Created kudu user
I20260809 19:23:05.901902 9581 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 19:23:05 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303383, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:06.305392 9581 mini_ranger_kms.cc:384] Added ranger policy
I20260809 19:23:06.305594 9581 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms:/tmp/dist-test-taskt8E36c/build/release/bin/postgresql.jar:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskt8E36c/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 19:23:06.305619 9581 mini_ranger_kms.cc:240] Using host: 127.9.91.84
I20260809 19:23:06.310281 9581 mini_ranger_kms.cc:276] Ranger KMS PID: 28044
I20260809 19:23:06.310341 9581 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.91.84:42359
Aug 09, 2026 7:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:23:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 19:23:08 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303388, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.91.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:08.753672 9581 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.101s user 0.006s sys 0.138s
I20260809 19:23:08.753775 9581 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 19:23:08.753824 9581 mini_ranger_kms.cc:401] 127.9.91.84:42359/kms/v1/keys
Aug 09 19:23:09 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303388, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.91.84@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
Aug 09 19:23:09 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303388, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.91.84@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
Aug 09 19:23:11 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:23:11.589339 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37937
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:37937
--ranger_config_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:11.703027 28122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:11.703212 28122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:11.703243 28122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:11.704386 28122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:11.704433 28122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:11.704456 28122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:11.704474 28122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:11.704492 28122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:11.706022 28122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.91.84:42359/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:37937
--ranger_config_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:11.706244 28122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:11.706437 28122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:11.711908 28133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:11.711966 28132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:11.711947 28139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:11.712213 28122 server_base.cc:1034] running on GCE node
I20260809 19:23:11.712445 28122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:11.712738 28122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:11.713893 28122 hybrid_clock.cc:648] HybridClock initialized: now 1786303391713876 us; error 33 us; skew 500 ppm
Aug 09 19:23:11 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303391, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:11.720820 28122 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:11.721216 28122 webserver.cc:460] Webserver started at http://127.9.91.126:33649/ using document root <none> and password file <none>
I20260809 19:23:11.721417 28122 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:11.721453 28122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:11.721563 28122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:23:11 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303391, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:11.764621 28122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "b671ede5e63444b99ec13fb4c7ff13d9"
format_stamp: "Formatted at 2026-08-09 19:23:11 on dist-test-slave-1vmg"
server_key: "290cfe2f84dcb84b32bc97a41645abda"
server_key_iv: "9d795583d2d51dd4eb73b93bed6d41e8"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:11.764997 28122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "b671ede5e63444b99ec13fb4c7ff13d9"
format_stamp: "Formatted at 2026-08-09 19:23:11 on dist-test-slave-1vmg"
server_key: "290cfe2f84dcb84b32bc97a41645abda"
server_key_iv: "9d795583d2d51dd4eb73b93bed6d41e8"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:11.766296 28122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:23:11.776752 28149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:11.776913 28122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:23:11.777019 28122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "b671ede5e63444b99ec13fb4c7ff13d9"
format_stamp: "Formatted at 2026-08-09 19:23:11 on dist-test-slave-1vmg"
server_key: "290cfe2f84dcb84b32bc97a41645abda"
server_key_iv: "9d795583d2d51dd4eb73b93bed6d41e8"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:11.777092 28122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:11.803236 28122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:11.804873 28122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:11.804997 28122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:11.809573 28122 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:37937
I20260809 19:23:11.809634 28222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:37937 every 8 connection(s)
I20260809 19:23:11.809947 28122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:11.810596 28224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:11.812779 28224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Bootstrap starting.
I20260809 19:23:11.813316 28224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:11.813751 28224 log.cc:824] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:11.814388 28224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: No bootstrap required, opened a new log
I20260809 19:23:11.814680 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 28122
I20260809 19:23:11.814774 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:23:11.814981 9581 external_mini_cluster.cc:1428] Setting key 0326d405aef692611896bd8e3c6f81f0
I20260809 19:23:11.816906 28224 raft_consensus.cc:348] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:11.817109 28224 raft_consensus.cc:374] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:11.817181 28224 raft_consensus.cc:729] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b671ede5e63444b99ec13fb4c7ff13d9, State: Initialized, Role: FOLLOWER
I20260809 19:23:11.817329 28224 consensus_queue.cc:260] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:11.817401 28224 raft_consensus.cc:388] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:23:11.817431 28224 raft_consensus.cc:482] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:23:11.817490 28224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:11.818048 28224 raft_consensus.cc:504] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:11.818166 28224 leader_election.cc:302] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b671ede5e63444b99ec13fb4c7ff13d9; no voters:
I20260809 19:23:11.818351 28224 leader_election.cc:288] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:23:11.818401 28230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:11.818627 28230 raft_consensus.cc:686] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 LEADER]: Becoming Leader. State: Replica: b671ede5e63444b99ec13fb4c7ff13d9, State: Running, Role: LEADER
I20260809 19:23:11.818684 28224 sys_catalog.cc:564] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:11.818801 28230 consensus_queue.cc:237] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:11.819205 28230 sys_catalog.cc:455] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } } }
I20260809 19:23:11.819334 28230 sys_catalog.cc:458] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:11.819514 28230 sys_catalog.cc:455] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b671ede5e63444b99ec13fb4c7ff13d9. Latest consensus state: current_term: 1 leader_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } } }
I20260809 19:23:11.819617 28230 sys_catalog.cc:458] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:11.819774 28245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:23:11.820071 28224 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:23:11.820346 28245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:23:11 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:11.826000 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:11.815460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.9.91.126:37937)
0809 19:23:11.815648 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:11.815657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:11.815690 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:23:11.816886 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:11.816895 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:11.816960 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:11.817110 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:11.817912 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:11.819313 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:11.820404 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:11.820831 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:11.820925 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:11.823111 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:11.823115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:11.823115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:11.823124 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:11.824490 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:11.824807 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:11.824810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:11.824811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:11.824896 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:11.825077 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:11.825077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:11.825078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:11.825297 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:11.825433 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:11.825528 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 19:23:11.825636 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260809 19:23:11.830842 28245 catalog_manager.cc:1347] Generated new cluster ID: a12752a2109b43b583b4e19be7812605
I20260809 19:23:11.831007 28245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:23:11.857342 28245 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:23:11.858135 28245 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:23:11.864630 28245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Generated new TSK 0
I20260809 19:23:11.864926 28245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:23:12.541430 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:37937
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:12.626854 28297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:12.627075 28297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:12.627107 28297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:12.628222 28297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:12.628263 28297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:12.628297 28297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:23:12.629726 28297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.91.84:42359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:12.629951 28297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:12.630168 28297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:12.634667 28304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:12.634893 28305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:12.635210 28307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:12.635607 28297 server_base.cc:1034] running on GCE node
I20260809 19:23:12.635797 28297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:12.636015 28297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:12.637176 28297 hybrid_clock.cc:648] HybridClock initialized: now 1786303392637160 us; error 27 us; skew 500 ppm
Aug 09 19:23:12 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303392, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:12.640689 28297 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:23:12.641161 28297 webserver.cc:460] Webserver started at http://127.9.91.65:46155/ using document root <none> and password file <none>
I20260809 19:23:12.641396 28297 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:12.641469 28297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:12.641610 28297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:23:12 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303392, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:12.655336 28297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "87999696ea4440dcbdacd7067d45e25b"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "908d6381c6d5cdb84f7d7bef686c79f2"
server_key_iv: "0521e57a61f19052c2b73f751c9ebd60"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:12.655853 28297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "87999696ea4440dcbdacd7067d45e25b"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "908d6381c6d5cdb84f7d7bef686c79f2"
server_key_iv: "0521e57a61f19052c2b73f751c9ebd60"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:12.657866 28297 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 19:23:12.670202 28315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:12.670395 28297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:12.670496 28297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "87999696ea4440dcbdacd7067d45e25b"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "908d6381c6d5cdb84f7d7bef686c79f2"
server_key_iv: "0521e57a61f19052c2b73f751c9ebd60"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:12.670573 28297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:12.684283 28297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:12.686486 28297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:12.686668 28297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:12.686964 28297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:12.687369 28297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:12.687460 28297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:12.687558 28297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:12.687638 28297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:12.694386 28297 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:46569
I20260809 19:23:12.694664 28481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:46569 every 8 connection(s)
I20260809 19:23:12.695134 28297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:23:12.695909 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 28297
I20260809 19:23:12.696013 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:23:12.696218 9581 external_mini_cluster.cc:1428] Setting key baa749abecffe792655751c5424653d8
Aug 09 19:23:12 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303392, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
I20260809 19:23:12.712586 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:12.696839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:58685 (local address 127.9.91.126:37937)
0809 19:23:12.703033 (+ 6194us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:12.703035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:12.703045 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:23:12.703070 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:12.703071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:12.703084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:12.703162 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:12.703852 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.704641 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.706068 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.706378 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.706444 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:12.708983 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:12.708984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:12.708985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:12.708992 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:12.711253 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:12.711638 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:12.711639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:12.711640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:12.711708 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:12.711976 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:12.711977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:12.711978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:12.712085 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:12.712139 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:12.712407 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 19:23:12.712492 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6156,"thread_start_us":81,"threads_started":1}
I20260809 19:23:12.712725 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:12.696523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37937 (local address 127.9.91.65:58685)
0809 19:23:12.696690 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:12.696736 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:12.699261 (+ 2525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:12.703260 (+ 3999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:12.703265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:12.703484 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:12.703736 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:12.703742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.704748 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.704750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:12.705958 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:12.705962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.706486 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.706488 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:12.706636 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:12.707192 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:12.707202 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:12.708876 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:12.711333 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:12.711339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:12.711342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:12.711545 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:12.711781 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:12.711782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:12.711782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:12.711906 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:12.712148 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:12.712154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:12.712290 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 19:23:12.712375 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 19:23:12.712506 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260809 19:23:12.713191 28483 heartbeater.cc:347] Connected to a master server at 127.9.91.126:37937
I20260809 19:23:12.713294 28483 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
I20260809 19:23:12.713475 28483 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:12.714003 28175 ts_manager.cc:194] Registered new tserver with Master: 87999696ea4440dcbdacd7067d45e25b (127.9.91.65:46569)
I20260809 19:23:12.714576 28175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:58685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:23:12.727506 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:37937
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:12.843806 28498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:12.844158 28498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:12.844262 28498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:12.845826 28498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:12.845932 28498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:12.846040 28498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:23:12.848129 28498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.91.84:42359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:12.848376 28498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:12.848671 28498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:12.854719 28517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:12.854715 28515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:12.855146 28511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:12.856022 28498 server_base.cc:1034] running on GCE node
I20260809 19:23:12.856231 28498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:12.856510 28498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:12.857697 28498 hybrid_clock.cc:648] HybridClock initialized: now 1786303392857675 us; error 33 us; skew 500 ppm
Aug 09 19:23:12 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303392, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:12.860931 28498 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:23:12.861407 28498 webserver.cc:460] Webserver started at http://127.9.91.66:41941/ using document root <none> and password file <none>
I20260809 19:23:12.861690 28498 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:12.861750 28498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:12.861899 28498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:23:12 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303392, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:12.877022 28498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "86eca227eafe4bfe8579b346511f7c12"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "9a7c85de55472a2d881a108dfac6a679"
server_key_iv: "77bf8d5721a80133dffe813447173d55"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:12.877549 28498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "86eca227eafe4bfe8579b346511f7c12"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "9a7c85de55472a2d881a108dfac6a679"
server_key_iv: "77bf8d5721a80133dffe813447173d55"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:12.883265 28498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 19:23:12.903080 28525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:12.903337 28498 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260809 19:23:12.903491 28498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "86eca227eafe4bfe8579b346511f7c12"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "9a7c85de55472a2d881a108dfac6a679"
server_key_iv: "77bf8d5721a80133dffe813447173d55"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:12.903636 28498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 19:23:12 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303392, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:12.933298 28498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:12.935506 28498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:12.935652 28498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:12.935864 28498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:12.936210 28498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:12.936259 28498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:12.936287 28498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:12.936308 28498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:12.942386 28498 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:41421
I20260809 19:23:12.942778 28498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:23:12.945227 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 28498
I20260809 19:23:12.945338 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:23:12.945561 9581 external_mini_cluster.cc:1428] Setting key b056aff47f6d0007a2303aa7d0ec8c53
I20260809 19:23:12.945734 28690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:41421 every 8 connection(s)
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Aug 09 19:23:12 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303392, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
I20260809 19:23:12.976742 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:12.946638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:40983 (local address 127.9.91.126:37937)
0809 19:23:12.946697 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:12.946699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:12.958296 (+ 11597us) server_negotiation.cc:407] Connection header received
0809 19:23:12.959053 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:12.959054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:12.959069 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:12.959101 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:12.964210 (+ 5109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.964982 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.966500 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.966837 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.966877 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:12.973130 (+ 6253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:12.973131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:12.973132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:12.973140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:12.974887 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:12.975522 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:12.975523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:12.975523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:12.975587 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:12.975902 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:12.975904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:12.975904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:12.976043 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:12.976113 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:12.976532 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 19:23:12.976626 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:23:12.977449 28693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:12.946373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37937 (local address 127.9.91.66:40983)
0809 19:23:12.958143 (+ 11770us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:12.958196 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:12.958890 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:12.959300 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:12.959304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:12.959552 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:12.959881 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:12.959888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.965108 (+ 5220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.965109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:12.966394 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:12.966397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:12.967003 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:12.967004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:12.967141 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:12.967673 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:12.967684 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:12.970837 (+ 3153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:12.975088 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:12.975094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:12.975097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:12.975425 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:12.975658 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:12.975658 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:12.975659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:12.975802 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:12.976191 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:12.976198 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:12.976373 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 19:23:12.977060 (+ 687us) client_negotiation.cc:240] Negotiation successful
0809 19:23:12.977221 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11720,"thread_start_us":85,"threads_started":1}
I20260809 19:23:12.978114 28691 heartbeater.cc:347] Connected to a master server at 127.9.91.126:37937
I20260809 19:23:12.978211 28691 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:12.978403 28691 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:12.978817 28175 ts_manager.cc:194] Registered new tserver with Master: 86eca227eafe4bfe8579b346511f7c12 (127.9.91.66:41421)
I20260809 19:23:12.979305 28175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:40983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:23:12.995615 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:37937
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:13.126029 28702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:13.126298 28702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:13.126334 28702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:13.127969 28702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:13.128027 28702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:13.128067 28702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:23:13.130151 28702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.91.84:42359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:13.130407 28702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:13.130661 28702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:13.137313 28718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:13.137313 28716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:13.137683 28714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.138747 28702 server_base.cc:1034] running on GCE node
I20260809 19:23:13.138986 28702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:13.139249 28702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:13.140419 28702 hybrid_clock.cc:648] HybridClock initialized: now 1786303393140386 us; error 45 us; skew 500 ppm
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:13.143707 28702 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:23:13.144193 28702 webserver.cc:460] Webserver started at http://127.9.91.67:38573/ using document root <none> and password file <none>
I20260809 19:23:13.144479 28702 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:13.144569 28702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:13.144709 28702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:13.159755 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "3d901e513aac4ee0a50294ccd1babeff"
format_stamp: "Formatted at 2026-08-09 19:23:13 on dist-test-slave-1vmg"
server_key: "ac25177ae747bfc8186ff17875461d3c"
server_key_iv: "f87869aeda869ff498c51752426d6a83"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:13.160219 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "3d901e513aac4ee0a50294ccd1babeff"
format_stamp: "Formatted at 2026-08-09 19:23:13 on dist-test-slave-1vmg"
server_key: "ac25177ae747bfc8186ff17875461d3c"
server_key_iv: "f87869aeda869ff498c51752426d6a83"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:13.172470 28702 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.000s sys 0.003s
I20260809 19:23:13.188920 28726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.189100 28702 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 19:23:13.189193 28702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "3d901e513aac4ee0a50294ccd1babeff"
format_stamp: "Formatted at 2026-08-09 19:23:13 on dist-test-slave-1vmg"
server_key: "ac25177ae747bfc8186ff17875461d3c"
server_key_iv: "f87869aeda869ff498c51752426d6a83"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:13.189306 28702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:13.209390 28702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:13.211829 28702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:13.211988 28702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:13.212196 28702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:13.212560 28702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:13.212613 28702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.212651 28702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:13.212680 28702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.222216 28702 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:44933
I20260809 19:23:13.222681 28702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:23:13.223455 28893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:44933 every 8 connection(s)
I20260809 19:23:13.228006 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 28702
I20260809 19:23:13.228124 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:23:13.228307 9581 external_mini_cluster.cc:1428] Setting key 860f3d50cd6d95e23245db525f6c3716
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:13.234885 28487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.224458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:53561 (local address 127.9.91.126:37937)
0809 19:23:13.224518 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:13.224520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:13.224880 (+ 360us) server_negotiation.cc:407] Connection header received
0809 19:23:13.225584 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:13.225586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:13.225599 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:13.225670 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:13.226361 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.227099 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.228631 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.228919 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.228987 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:13.231606 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:13.231608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:13.231608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:13.231615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:13.233254 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:13.233779 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:13.233780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:13.233781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:13.233846 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:13.234145 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:13.234146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:13.234147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:13.234287 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:13.234353 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:13.234703 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 19:23:13.234791 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:23:13.235162 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.224142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37937 (local address 127.9.91.67:53561)
0809 19:23:13.224752 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:13.224795 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:13.225439 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:13.225741 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:13.225746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:13.225986 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:13.226250 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:13.226256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.227210 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.227212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:13.228461 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:13.228464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.229068 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.229069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:13.229236 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:13.229808 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:13.229818 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:13.231509 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:13.233349 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:13.233356 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:13.233359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:13.233684 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:13.233913 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:13.233914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:13.233914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:13.234072 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:13.234358 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:13.234364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:13.234548 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 19:23:13.234675 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 19:23:13.234828 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":88,"threads_started":1}
I20260809 19:23:13.235689 28894 heartbeater.cc:347] Connected to a master server at 127.9.91.126:37937
I20260809 19:23:13.235801 28894 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:13.235992 28894 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:13.236423 28175 ts_manager.cc:194] Registered new tserver with Master: 3d901e513aac4ee0a50294ccd1babeff (127.9.91.67:44933)
I20260809 19:23:13.236871 28175 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:53561
I20260809 19:23:13.239991 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:23:13.240836 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 28297
I20260809 19:23:13.251550 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 28498
I20260809 19:23:13.262475 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 28702
I20260809 19:23:13.276274 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 28122
I20260809 19:23:13.286441 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37937
--webserver_interface=127.9.91.126
--webserver_port=33649
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:37937
--ranger_config_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:13.371795 28897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:13.371973 28897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:13.371994 28897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:13.373073 28897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:13.373112 28897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:13.373124 28897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:13.373147 28897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:13.373168 28897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:13.374650 28897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:37937
--ranger_config_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=33649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:13.374854 28897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:13.375067 28897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:13.379449 28905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:13.379458 28908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.379489 28897 server_base.cc:1034] running on GCE node
W20260809 19:23:13.379601 28906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.379830 28897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:13.380110 28897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:13.381253 28897 hybrid_clock.cc:648] HybridClock initialized: now 1786303393381237 us; error 27 us; skew 500 ppm
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:13.383526 28897 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:13.383919 28897 webserver.cc:460] Webserver started at http://127.9.91.126:33649/ using document root <none> and password file <none>
I20260809 19:23:13.384109 28897 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:13.384161 28897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:13.418118 28897 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:13.418805 28920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.418977 28897 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.419065 28897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "b671ede5e63444b99ec13fb4c7ff13d9"
format_stamp: "Formatted at 2026-08-09 19:23:11 on dist-test-slave-1vmg"
server_key: "290cfe2f84dcb84b32bc97a41645abda"
server_key_iv: "9d795583d2d51dd4eb73b93bed6d41e8"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:13.419366 28897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:13.438877 28897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:13.440245 28897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:13.440369 28897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:13.444151 28897 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:37937
I20260809 19:23:13.444202 28992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:37937 every 8 connection(s)
I20260809 19:23:13.444533 28897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:13.445629 28993 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:23:13.446233 28993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Bootstrap starting.
I20260809 19:23:13.450299 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 28897
I20260809 19:23:13.450560 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:46569
--local_ip_for_outbound_sockets=127.9.91.65
--tserver_master_addrs=127.9.91.126:37937
--webserver_port=46155
--webserver_interface=127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:13.453665 28993 log.cc:824] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:13.455292 28993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:23:13.455520 28993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Bootstrap complete.
I20260809 19:23:13.457536 28993 raft_consensus.cc:348] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:13.457755 28993 raft_consensus.cc:729] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b671ede5e63444b99ec13fb4c7ff13d9, State: Initialized, Role: FOLLOWER
I20260809 19:23:13.457904 28993 consensus_queue.cc:260] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:13.457974 28993 raft_consensus.cc:388] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:23:13.458022 28993 raft_consensus.cc:482] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:23:13.458067 28993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:23:13.459007 28993 raft_consensus.cc:504] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:13.459101 28993 leader_election.cc:302] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b671ede5e63444b99ec13fb4c7ff13d9; no voters:
I20260809 19:23:13.459259 28993 leader_election.cc:288] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:23:13.459316 28997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:23:13.459446 28997 raft_consensus.cc:686] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [term 2 LEADER]: Becoming Leader. State: Replica: b671ede5e63444b99ec13fb4c7ff13d9, State: Running, Role: LEADER
I20260809 19:23:13.459484 28993 sys_catalog.cc:564] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:13.459550 28997 consensus_queue.cc:237] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } }
I20260809 19:23:13.459832 28997 sys_catalog.cc:455] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b671ede5e63444b99ec13fb4c7ff13d9. Latest consensus state: current_term: 2 leader_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } } }
I20260809 19:23:13.459913 28997 sys_catalog.cc:458] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:13.460067 28998 sys_catalog.cc:455] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b671ede5e63444b99ec13fb4c7ff13d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37937 } } }
I20260809 19:23:13.460139 28998 sys_catalog.cc:458] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:13.460391 29006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:23:13.460882 29006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:23:13.461025 29006 catalog_manager.cc:1259] Loaded cluster ID: a12752a2109b43b583b4e19be7812605
I20260809 19:23:13.461067 29006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:23:13.461225 28993 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 19:23:13.464319 29006 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:23:13.464941 29006 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Loaded TSK: 0
I20260809 19:23:13.465233 29006 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:23:13.555308 28995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:13.555502 28995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:13.555533 28995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:13.556663 28995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:13.556703 28995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:13.556737 28995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:23:13.558185 28995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:46569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=46155
--tserver_master_addrs=127.9.91.126:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:13.558414 28995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:13.558625 28995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:13.565361 29039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:13.565840 29044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:13.566360 29042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.566617 28995 server_base.cc:1034] running on GCE node
I20260809 19:23:13.566817 28995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:13.567090 28995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:13.568259 28995 hybrid_clock.cc:648] HybridClock initialized: now 1786303393568232 us; error 39 us; skew 500 ppm
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:13.570990 28995 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:23:13.571419 28995 webserver.cc:460] Webserver started at http://127.9.91.65:46155/ using document root <none> and password file <none>
I20260809 19:23:13.571631 28995 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:13.571694 28995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:13.599936 28995 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:13.607021 29059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.607203 28995 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260809 19:23:13.607292 28995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "87999696ea4440dcbdacd7067d45e25b"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "908d6381c6d5cdb84f7d7bef686c79f2"
server_key_iv: "0521e57a61f19052c2b73f751c9ebd60"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:13.607658 28995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:13.622249 28995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:13.624454 28995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:13.624615 28995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:13.624815 28995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:13.625154 28995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:13.625191 28995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.625212 28995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:13.625223 28995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.632063 28995 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:46569
I20260809 19:23:13.632140 29220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:46569 every 8 connection(s)
I20260809 19:23:13.632452 28995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:23:13.638582 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 28995
I20260809 19:23:13.638899 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:41421
--local_ip_for_outbound_sockets=127.9.91.66
--tserver_master_addrs=127.9.91.126:37937
--webserver_port=41941
--webserver_interface=127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:13.646392 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.633173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37937 (local address 127.9.91.65:46085)
0809 19:23:13.633356 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:13.633401 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:13.633872 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:13.634288 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:13.634292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:13.634461 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:13.634639 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:13.634644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.635731 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.635733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:13.636698 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:13.636701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.637379 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.637380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:13.637508 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:13.638001 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:13.638011 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:13.639779 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:13.644761 (+ 4982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:13.644768 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:13.644771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:13.645041 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:13.645338 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:13.645339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:13.645340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:13.645462 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:13.645801 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:13.645807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:13.645950 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 19:23:13.646047 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 19:23:13.646182 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260809 19:23:13.647292 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.633239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:46085 (local address 127.9.91.126:37937)
0809 19:23:13.633441 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:13.633449 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:13.633482 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:23:13.634079 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:13.634084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:13.634129 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:13.634241 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:13.634750 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.635633 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.636783 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.637277 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.637388 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:13.639886 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:13.639891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:13.639891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:13.639903 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:13.644659 (+ 4756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:13.645147 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:13.645150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:13.645150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:13.645262 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:13.645524 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:13.645524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:13.645524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:13.645729 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:13.645797 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:13.646981 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0809 19:23:13.647096 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":83,"threads_started":1}
I20260809 19:23:13.648311 29222 heartbeater.cc:347] Connected to a master server at 127.9.91.126:37937
I20260809 19:23:13.648399 29222 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:13.648579 29222 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
W20260809 19:23:13.649477 29222 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:37937 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:23:13.649544 29222 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:13.649614 29222 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:13.650084 29222 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:13.650177 29222 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
W20260809 19:23:13.650534 29222 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:23:13.748373 29229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:13.748611 29229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:13.748649 29229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:13.750139 29229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:13.750195 29229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:13.750232 29229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:23:13.752300 29229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:41421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=41941
--tserver_master_addrs=127.9.91.126:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:13.752521 29229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:13.752779 29229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:13.757622 29236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.757889 29229 server_base.cc:1034] running on GCE node
W20260809 19:23:13.758019 29238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:13.758147 29235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.758383 29229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:13.758615 29229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:13.759781 29229 hybrid_clock.cc:648] HybridClock initialized: now 1786303393759752 us; error 48 us; skew 500 ppm
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:13.763120 29229 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:23:13.763640 29229 webserver.cc:460] Webserver started at http://127.9.91.66:41941/ using document root <none> and password file <none>
I20260809 19:23:13.763893 29229 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:13.763959 29229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:13.778892 29229 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:13.779847 29246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.780025 29229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 19:23:13.780125 29229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "86eca227eafe4bfe8579b346511f7c12"
format_stamp: "Formatted at 2026-08-09 19:23:12 on dist-test-slave-1vmg"
server_key: "9a7c85de55472a2d881a108dfac6a679"
server_key_iv: "77bf8d5721a80133dffe813447173d55"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:13.780457 29229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:13.800762 29229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:13.802969 29229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:13.803094 29229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:13.803283 29229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:13.803604 29229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:13.803651 29229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.803682 29229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:13.803709 29229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.811406 29229 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:41421
I20260809 19:23:13.811801 29229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:23:13.812407 29418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:41421 every 8 connection(s)
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:13.820334 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 29229
I20260809 19:23:13.820644 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.91.84:42359/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:44933
--local_ip_for_outbound_sockets=127.9.91.67
--tserver_master_addrs=127.9.91.126:37937
--webserver_port=38573
--webserver_interface=127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:13.825735 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.812781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:47863 (local address 127.9.91.126:37937)
0809 19:23:13.812858 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:13.812860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:13.812874 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:23:13.813304 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:13.813305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:13.813323 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:13.813392 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:13.814138 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.814887 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.816335 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.816641 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.816715 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:13.819227 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:13.819228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:13.819228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:13.819236 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:13.823958 (+ 4722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:13.824540 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:13.824541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:13.824542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:13.824617 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:13.824909 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:13.824910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:13.824911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:13.825063 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:13.825154 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:13.825479 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 19:23:13.825579 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 19:23:13.826043 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.812542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37937 (local address 127.9.91.66:47863)
0809 19:23:13.812694 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:13.812733 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:13.813199 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:13.813532 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:13.813537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:13.813749 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:13.814016 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:13.814023 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.815036 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.815037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:13.816232 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:13.816236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:13.816767 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:13.816769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:13.816908 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:13.817440 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:13.817450 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:13.819106 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:13.824147 (+ 5041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:13.824154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:13.824155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:13.824444 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:13.824700 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:13.824700 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:13.824701 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:13.824836 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:13.825168 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:13.825174 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:13.825330 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 19:23:13.825446 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 19:23:13.825615 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260809 19:23:13.826563 29419 heartbeater.cc:347] Connected to a master server at 127.9.91.126:37937
I20260809 19:23:13.826664 29419 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:13.826858 29419 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
W20260809 19:23:13.827634 29419 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:37937 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:23:13.827713 29419 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:13.827785 29419 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:13.828186 29419 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:13.828276 29419 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
W20260809 19:23:13.828686 29419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:23:13.919090 29423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:13.919279 29423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:13.919312 29423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:13.920683 29423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:13.920732 29423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:13.920769 29423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:23:13.922336 29423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.91.84:42359/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:44933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=38573
--tserver_master_addrs=127.9.91.126:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:13.922565 29423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:13.922811 29423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:13.927752 29431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:13.927757 29435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.927973 29423 server_base.cc:1034] running on GCE node
W20260809 19:23:13.927786 29432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:13.928283 29423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:13.928526 29423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:13.929675 29423 hybrid_clock.cc:648] HybridClock initialized: now 1786303393929663 us; error 28 us; skew 500 ppm
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:13.932210 29423 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:23:13.932643 29423 webserver.cc:460] Webserver started at http://127.9.91.67:38573/ using document root <none> and password file <none>
I20260809 19:23:13.932873 29423 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:13.932941 29423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 19:23:13 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:13.953562 29423 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:13.954795 29451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.955018 29423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:13.955135 29423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "3d901e513aac4ee0a50294ccd1babeff"
format_stamp: "Formatted at 2026-08-09 19:23:13 on dist-test-slave-1vmg"
server_key: "ac25177ae747bfc8186ff17875461d3c"
server_key_iv: "f87869aeda869ff498c51752426d6a83"
server_key_version: "kuduclusterkey@0"
I20260809 19:23:13.955540 29423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:13.984702 29423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:13.986482 29423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:13.986629 29423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:13.986840 29423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:13.987190 29423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:13.987231 29423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.987263 29423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:13.987295 29423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:13.994292 29423 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:44933
I20260809 19:23:13.994374 29627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:44933 every 8 connection(s)
I20260809 19:23:13.994740 29423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:23:13.998298 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 29423
Aug 09 19:23:14 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303393, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:14.009003 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.998756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46198 (local address 127.9.91.126:37937)
0809 19:23:13.998804 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:13.998806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:13.999820 (+ 1014us) server_negotiation.cc:407] Connection header received
0809 19:23:13.999852 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:13.999853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:13.999865 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:13.999910 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:14.000743 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:14.001472 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:14.002622 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:14.002948 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:14.003001 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:14.004484 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:14.004485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:14.004486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:14.004493 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:14.006394 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:14.007562 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:14.007563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:14.007563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:14.007639 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:14.007835 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:14.007835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:14.007836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:14.007955 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:14.008328 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:14.008807 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 19:23:14.008924 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:23:14.010878 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.999373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:58667 (local address 127.9.91.126:37937)
0809 19:23:14.002001 (+ 2628us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:14.002003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:14.002011 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:14.002045 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:14.002046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:14.002057 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:14.002127 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:14.002839 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:14.003593 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:14.005037 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:14.005331 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:14.005397 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:14.007749 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:14.007750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:14.007751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:14.007758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:14.009363 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:14.009968 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:14.009969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:14.009969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:14.010054 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:14.010327 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:14.010328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:14.010328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:14.010421 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:14.010468 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:14.010760 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 19:23:14.010819 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2606,"thread_start_us":67,"threads_started":1}
I20260809 19:23:14.011139 29633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:13.999368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:37937 (local address 127.9.91.67:58667)
0809 19:23:14.001187 (+ 1819us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:14.001233 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:14.001826 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:14.002253 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:14.002258 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:14.002456 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:14.002721 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:14.002728 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:14.003706 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:14.003707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:14.004929 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:14.004932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:14.005480 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:14.005481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:14.005624 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:14.006093 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:14.006102 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:14.007642 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:14.009449 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:14.009455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:14.009457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:14.009892 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:14.010123 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:14.010124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:14.010124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:14.010267 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:14.010480 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:14.010485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:14.010636 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 19:23:14.010751 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 19:23:14.010890 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1774,"thread_start_us":74,"threads_started":1}
I20260809 19:23:14.011618 29628 heartbeater.cc:347] Connected to a master server at 127.9.91.126:37937
I20260809 19:23:14.011722 29628 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:14.011904 29628 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
W20260809 19:23:14.012614 29628 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:37937 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 19:23:14.012686 29628 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:14.012739 29628 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:14.013085 29628 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:14.013154 29628 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
W20260809 19:23:14.013479 29628 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 19:23:14 dist-test-slave-1vmg krb5kdc[27430](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786303394, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:14.650707 29222 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:14.650862 29222 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:14.651244 28941 ts_manager.cc:194] Registered new tserver with Master: 87999696ea4440dcbdacd7067d45e25b (127.9.91.65:46569)
I20260809 19:23:14.652292 28941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:46085
I20260809 19:23:14.828866 29419 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:14.829007 29419 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:14.829298 28941 ts_manager.cc:194] Registered new tserver with Master: 86eca227eafe4bfe8579b346511f7c12 (127.9.91.66:41421)
I20260809 19:23:14.829830 28941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:47863
Aug 09 19:23:14 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303394, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
I20260809 19:23:15.013653 29628 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:15.013815 29628 heartbeater.cc:511] Master 127.9.91.126:37937 requested a full tablet report, sending...
I20260809 19:23:15.014147 28941 ts_manager.cc:194] Registered new tserver with Master: 3d901e513aac4ee0a50294ccd1babeff (127.9.91.67:44933)
I20260809 19:23:15.014639 28941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:58667
I20260809 19:23:15.016863 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:23:15.027347 29657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.017811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.9.91.126:37937)
0809 19:23:15.018158 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:15.018160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:15.018316 (+ 156us) server_negotiation.cc:407] Connection header received
0809 19:23:15.018339 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:15.018340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:15.018353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:15.018420 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:15.019153 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.019888 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.022412 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.022706 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.022783 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.024397 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:15.024398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:15.024399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:15.024406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:15.026120 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:15.026524 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:15.026525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:15.026525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:15.026593 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:15.026784 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:15.026785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:15.026785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:15.026937 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:15.027043 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:15.027159 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 19:23:15.027246 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":82,"threads_started":1}
I20260809 19:23:15.029417 28941 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:23:15.031808 28941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:23:15.042639 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.037046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:46569 (local address 127.0.0.1:52882)
0809 19:23:15.038266 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:15.038277 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:15.038335 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:15.038626 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:15.038627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:15.038629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:15.038672 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.038674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.039997 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.039998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.041163 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.041167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.042308 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.042309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.042390 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.042404 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:23:15.042484 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:23:15.042542 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1198,"thread_start_us":159,"threads_started":1}
I20260809 19:23:15.042699 29670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.037354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.9.91.67:44933)
0809 19:23:15.037913 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:15.037918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:15.037928 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:23:15.037963 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:15.037965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:15.037981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:15.038070 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:15.038243 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.039331 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.041159 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.042213 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.042266 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.042319 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:15.042587 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 19:23:15.042627 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":65,"threads_started":1}
I20260809 19:23:15.043120 29672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.037349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.9.91.65:46569)
0809 19:23:15.038200 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:15.038204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:15.038397 (+ 193us) server_negotiation.cc:407] Connection header received
0809 19:23:15.038434 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:15.038436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:15.038453 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:15.038540 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:15.038758 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.039882 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.041300 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.042069 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.042149 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.042267 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:15.042994 (+ 727us) server_negotiation.cc:299] Negotiation successful
0809 19:23:15.043037 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":89,"threads_started":1}
I20260809 19:23:15.042845 29669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.037044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:44933 (local address 127.0.0.1:50094)
0809 19:23:15.037664 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:15.037706 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:15.037772 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:15.038082 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:15.038085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:15.038087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:15.038148 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.038150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.039425 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.039426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.041049 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.041052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.042350 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.042351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.042416 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.042420 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:23:15.042467 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:23:15.042521 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":125,"threads_started":1}
I20260809 19:23:15.043816 29124 tablet_service.cc:1467] Processing CreateTablet for tablet 429411f21fb74d88a04f7c6074ceb7fb (DEFAULT_TABLE table=test-table [id=72d34afd67ca4f12859354d377aa2592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:15.044179 29124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 429411f21fb74d88a04f7c6074ceb7fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:15.047369 29525 tablet_service.cc:1467] Processing CreateTablet for tablet 429411f21fb74d88a04f7c6074ceb7fb (DEFAULT_TABLE table=test-table [id=72d34afd67ca4f12859354d377aa2592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:15.047704 29525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 429411f21fb74d88a04f7c6074ceb7fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:15.051755 29677 tablet_bootstrap.cc:492] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Bootstrap starting.
I20260809 19:23:15.052469 29677 tablet_bootstrap.cc:654] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:15.052803 29677 log.cc:824] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:15.053529 29677 tablet_bootstrap.cc:492] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: No bootstrap required, opened a new log
I20260809 19:23:15.053587 29677 ts_tablet_manager.cc:1397] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:23:15.055707 29677 raft_consensus.cc:348] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.055881 29677 raft_consensus.cc:374] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:15.055915 29677 raft_consensus.cc:729] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d901e513aac4ee0a50294ccd1babeff, State: Initialized, Role: FOLLOWER
I20260809 19:23:15.056028 29677 consensus_queue.cc:260] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.056236 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.037711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.9.91.66:41421)
0809 19:23:15.047076 (+ 9365us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:15.047081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:15.047095 (+ 14us) server_negotiation.cc:407] Connection header received
0809 19:23:15.052207 (+ 5112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:15.052211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:15.052228 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:15.052282 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:15.052516 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.053578 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.054868 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.055623 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.055672 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.055722 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:15.056101 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 19:23:15.056148 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9326,"thread_start_us":62,"threads_started":1}
I20260809 19:23:15.056282 29677 ts_tablet_manager.cc:1428] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:23:15.056414 29628 heartbeater.cc:503] Master 127.9.91.126:37937 was elected leader, sending a full tablet report...
I20260809 19:23:15.056640 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.037598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41421 (local address 127.0.0.1:47900)
0809 19:23:15.043159 (+ 5561us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:15.043296 (+ 137us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:15.052085 (+ 8789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:15.052399 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:15.052400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:15.052401 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:15.052433 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.052434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.053784 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.053785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.054767 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.054771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.055782 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.055783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.055841 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.055844 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:23:15.056515 (+ 671us) client_negotiation.cc:240] Negotiation successful
0809 19:23:15.056560 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5550,"thread_start_us":144,"threads_started":1}
I20260809 19:23:15.057525 29320 tablet_service.cc:1467] Processing CreateTablet for tablet 429411f21fb74d88a04f7c6074ceb7fb (DEFAULT_TABLE table=test-table [id=72d34afd67ca4f12859354d377aa2592]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:15.057860 29320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 429411f21fb74d88a04f7c6074ceb7fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:15.060817 29678 tablet_bootstrap.cc:492] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b: Bootstrap starting.
I20260809 19:23:15.061576 29678 tablet_bootstrap.cc:654] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:15.061941 29678 log.cc:824] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:15.062180 29683 tablet_bootstrap.cc:492] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Bootstrap starting.
I20260809 19:23:15.062805 29678 tablet_bootstrap.cc:492] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b: No bootstrap required, opened a new log
I20260809 19:23:15.062803 29683 tablet_bootstrap.cc:654] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:15.062872 29678 ts_tablet_manager.cc:1397] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:23:15.063154 29683 log.cc:824] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:15.063879 29683 tablet_bootstrap.cc:492] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: No bootstrap required, opened a new log
I20260809 19:23:15.063934 29683 ts_tablet_manager.cc:1397] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:23:15.065366 29678 raft_consensus.cc:348] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.065508 29678 raft_consensus.cc:374] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:15.065538 29678 raft_consensus.cc:729] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87999696ea4440dcbdacd7067d45e25b, State: Initialized, Role: FOLLOWER
I20260809 19:23:15.065858 29678 consensus_queue.cc:260] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.065943 29683 raft_consensus.cc:348] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.066107 29683 raft_consensus.cc:374] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:15.066156 29683 raft_consensus.cc:729] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86eca227eafe4bfe8579b346511f7c12, State: Initialized, Role: FOLLOWER
I20260809 19:23:15.066296 29678 ts_tablet_manager.cc:1428] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:23:15.066270 29683 consensus_queue.cc:260] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.066459 29222 heartbeater.cc:503] Master 127.9.91.126:37937 was elected leader, sending a full tablet report...
I20260809 19:23:15.066532 29683 ts_tablet_manager.cc:1428] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:23:15.066669 29419 heartbeater.cc:503] Master 127.9.91.126:37937 was elected leader, sending a full tablet report...
Aug 09 19:23:15 dist-test-slave-1vmg krb5kdc[27430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303394, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for HTTP/127.9.91.84@KRBTEST.COM
W20260809 19:23:15.133726 29223 tablet.cc:2369] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:23:15.246073 29630 tablet.cc:2369] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:23:15.290699 29687 raft_consensus.cc:482] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:15.290869 29687 raft_consensus.cc:504] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.291359 29687 leader_election.cc:288] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d901e513aac4ee0a50294ccd1babeff (127.9.91.67:44933), 86eca227eafe4bfe8579b346511f7c12 (127.9.91.66:41421)
I20260809 19:23:15.302599 29676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.292003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:41907 (local address 127.9.91.66:41421)
0809 19:23:15.292043 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:15.292045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:15.295081 (+ 3036us) server_negotiation.cc:407] Connection header received
0809 19:23:15.295173 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:15.295174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:15.295188 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:15.295214 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:15.295391 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.296071 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.301644 (+ 5573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.302186 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.302229 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.302239 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:15.302481 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 19:23:15.302520 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:23:15.303130 29692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.291751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41421 (local address 127.9.91.65:41907)
0809 19:23:15.295026 (+ 3275us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:15.295034 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:15.295105 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:15.295274 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:15.295274 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:15.295276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:15.295323 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.295324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.297110 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.297111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.298216 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.298220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.302342 (+ 4122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.302344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.302411 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.302414 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:23:15.302880 (+ 466us) client_negotiation.cc:240] Negotiation successful
0809 19:23:15.303010 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3260,"thread_start_us":67,"threads_started":1}
I20260809 19:23:15.303624 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429411f21fb74d88a04f7c6074ceb7fb" candidate_uuid: "87999696ea4440dcbdacd7067d45e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86eca227eafe4bfe8579b346511f7c12" is_pre_election: true
I20260809 19:23:15.303798 29338 raft_consensus.cc:2454] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87999696ea4440dcbdacd7067d45e25b in term 0.
I20260809 19:23:15.303953 29670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.291948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:47107 (local address 127.9.91.67:44933)
0809 19:23:15.291999 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:15.292002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:15.294899 (+ 2897us) server_negotiation.cc:407] Connection header received
0809 19:23:15.294993 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:15.294994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:15.295008 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:15.295071 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:15.295283 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.296016 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.303117 (+ 7101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.303620 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.303660 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.303669 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:15.303856 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 19:23:15.303893 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:23:15.304062 29062 leader_election.cc:302] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86eca227eafe4bfe8579b346511f7c12, 87999696ea4440dcbdacd7067d45e25b; no voters:
I20260809 19:23:15.304207 29687 raft_consensus.cc:2781] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:23:15.304286 29687 raft_consensus.cc:482] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:23:15.304313 29687 raft_consensus.cc:3037] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:15.305155 29687 raft_consensus.cc:504] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.305290 29687 leader_election.cc:288] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [CANDIDATE]: Term 1 election: Requested vote from peers 3d901e513aac4ee0a50294ccd1babeff (127.9.91.67:44933), 86eca227eafe4bfe8579b346511f7c12 (127.9.91.66:41421)
I20260809 19:23:15.305466 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429411f21fb74d88a04f7c6074ceb7fb" candidate_uuid: "87999696ea4440dcbdacd7067d45e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86eca227eafe4bfe8579b346511f7c12"
I20260809 19:23:15.305542 29338 raft_consensus.cc:3037] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:15.306349 29338 raft_consensus.cc:2454] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87999696ea4440dcbdacd7067d45e25b in term 1.
I20260809 19:23:15.306520 29062 leader_election.cc:302] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86eca227eafe4bfe8579b346511f7c12, 87999696ea4440dcbdacd7067d45e25b; no voters:
I20260809 19:23:15.306612 29687 raft_consensus.cc:2781] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:15.306828 29687 raft_consensus.cc:686] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [term 1 LEADER]: Becoming Leader. State: Replica: 87999696ea4440dcbdacd7067d45e25b, State: Running, Role: LEADER
I20260809 19:23:15.306955 29687 consensus_queue.cc:237] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } }
I20260809 19:23:15.307782 28940 catalog_manager.cc:5495] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b reported cstate change: term changed from 0 to 1, leader changed from <none> to 87999696ea4440dcbdacd7067d45e25b (127.9.91.65). New cstate: current_term: 1 leader_uuid: "87999696ea4440dcbdacd7067d45e25b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87999696ea4440dcbdacd7067d45e25b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 } health_report { overall_health: UNKNOWN } } }
I20260809 19:23:15.304193 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.291511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:44933 (local address 127.9.91.65:47107)
0809 19:23:15.294820 (+ 3309us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:15.294834 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:15.294934 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:15.295123 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:15.295124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:15.295127 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:15.295206 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.295208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.296127 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.296128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.297357 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:15.297359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.303719 (+ 6360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.303719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:15.303785 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.303788 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:23:15.304082 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 19:23:15.304122 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3284,"thread_start_us":121,"threads_started":1}
I20260809 19:23:15.309023 29554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429411f21fb74d88a04f7c6074ceb7fb" candidate_uuid: "87999696ea4440dcbdacd7067d45e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d901e513aac4ee0a50294ccd1babeff" is_pre_election: true
I20260809 19:23:15.309172 29554 raft_consensus.cc:2454] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87999696ea4440dcbdacd7067d45e25b in term 0.
I20260809 19:23:15.309283 29552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "429411f21fb74d88a04f7c6074ceb7fb" candidate_uuid: "87999696ea4440dcbdacd7067d45e25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d901e513aac4ee0a50294ccd1babeff"
I20260809 19:23:15.309342 29552 raft_consensus.cc:3037] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:15.310081 29552 raft_consensus.cc:2454] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87999696ea4440dcbdacd7067d45e25b in term 1.
W20260809 19:23:15.313138 29420 tablet.cc:2369] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:23:15.324733 29672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.321519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.9.91.65:46569)
0809 19:23:15.321556 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:15.321558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:15.321565 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:23:15.321596 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:15.321597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:15.321615 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:15.321673 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:23:15.321859 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.322581 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.323713 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:15.323988 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:15.324047 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:15.324247 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:23:15.324361 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:23:15.324428 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:15.324654 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 19:23:15.324672 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:23:15.333278 29338 raft_consensus.cc:1261] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 1 FOLLOWER]: Refusing update from remote peer 87999696ea4440dcbdacd7067d45e25b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:15.333316 29552 raft_consensus.cc:1261] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 1 FOLLOWER]: Refusing update from remote peer 87999696ea4440dcbdacd7067d45e25b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:15.333630 29694 consensus_queue.cc:1035] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d901e513aac4ee0a50294ccd1babeff" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:15.333789 29687 consensus_queue.cc:1035] T 429411f21fb74d88a04f7c6074ceb7fb P 87999696ea4440dcbdacd7067d45e25b [LEADER]: Connected to new peer: Peer: permanent_uuid: "86eca227eafe4bfe8579b346511f7c12" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:15.344393 29698 mvcc.cc:205] Tried to move back new op lower bound from 7316698707282296832 to 7316698707177885696. Current Snapshot: MvccSnapshot[applied={T|T < 7316698707282296832}]
I20260809 19:23:15.344576 29699 mvcc.cc:205] Tried to move back new op lower bound from 7316698707282296832 to 7316698707177885696. Current Snapshot: MvccSnapshot[applied={T|T < 7316698707282296832}]
I20260809 19:23:15.346428 28941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:15.346539 28941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:15.347272 28941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b671ede5e63444b99ec13fb4c7ff13d9: Sending DeleteTablet for 3 replicas of tablet 429411f21fb74d88a04f7c6074ceb7fb
I20260809 19:23:15.347648 29525 tablet_service.cc:1514] Processing DeleteTablet for tablet 429411f21fb74d88a04f7c6074ceb7fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:15 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50094
I20260809 19:23:15.348083 29320 tablet_service.cc:1514] Processing DeleteTablet for tablet 429411f21fb74d88a04f7c6074ceb7fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:15 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:47900
I20260809 19:23:15.348405 29124 tablet_service.cc:1514] Processing DeleteTablet for tablet 429411f21fb74d88a04f7c6074ceb7fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:15 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:52882
I20260809 19:23:15.348881 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 28995
I20260809 19:23:15.350967 29700 mvcc.cc:205] Tried to move back new op lower bound from 7316698707282296832 to 7316698707177885696. Current Snapshot: MvccSnapshot[applied={T|T < 7316698707282296832}]
I20260809 19:23:15.351111 29707 tablet_replica.cc:330] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: stopping tablet replica
I20260809 19:23:15.351197 29707 raft_consensus.cc:2229] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:15.351243 29707 raft_consensus.cc:2258] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:15.351352 29708 tablet_replica.cc:330] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: stopping tablet replica
I20260809 19:23:15.351415 29708 raft_consensus.cc:2229] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:15.351454 29708 raft_consensus.cc:2258] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 19:23:15.365403 28921 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:46569 (error 108)
I20260809 19:23:15.366042 29708 ts_tablet_manager.cc:1905] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:15.366086 29707 ts_tablet_manager.cc:1905] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:15.367740 29707 ts_tablet_manager.cc:1918] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:15.367758 29708 ts_tablet_manager.cc:1918] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:15.367815 29707 log.cc:1196] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/429411f21fb74d88a04f7c6074ceb7fb
I20260809 19:23:15.367818 29708 log.cc:1196] T 429411f21fb74d88a04f7c6074ceb7fb P 86eca227eafe4bfe8579b346511f7c12: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/429411f21fb74d88a04f7c6074ceb7fb
I20260809 19:23:15.368072 29707 ts_tablet_manager.cc:1939] T 429411f21fb74d88a04f7c6074ceb7fb P 3d901e513aac4ee0a50294ccd1babeff: Deleting consensus metadata
I20260809 19:23:15.368484 28925 catalog_manager.cc:4841] TS 3d901e513aac4ee0a50294ccd1babeff (127.9.91.67:44933): tablet 429411f21fb74d88a04f7c6074ceb7fb (table test-table [id=72d34afd67ca4f12859354d377aa2592]) successfully deleted
I20260809 19:23:15.368975 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 29229
I20260809 19:23:15.378234 29668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:15.378111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:46569 (local address 127.0.0.1:52892)
0809 19:23:15.378155 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:15.378200 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:46569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:23:15.378355 28921 catalog_manager.cc:4584] TS 87999696ea4440dcbdacd7067d45e25b (127.9.91.65:46569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 429411f21fb74d88a04f7c6074ceb7fb: Network error: Client connection negotiation failed: client connection to 127.9.91.65:46569: connect: Connection refused (error 111)
W20260809 19:23:15.381012 29668 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:23:15.380894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41421 (local address 127.0.0.1:47904)
0809 19:23:15.380938 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:15.380965 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.66:41421: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:23:15.381201 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 29423
I20260809 19:23:15.391166 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 28897
I20260809 19:23:15.402258 9581 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 19:23:15.747516 9581 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 19:23:15.747605 9581 mini_ranger.cc:66] Stopping Ranger...
I20260809 19:23:16.091583 9581 mini_ranger.cc:68] Stopped Ranger
2026-08-09 19:23:16.091 UTC [27478] LOG: received smart shutdown request
2026-08-09 19:23:16.095 UTC [27478] LOG: background worker "logical replication launcher" (PID 27494) exited with exit code 1
2026-08-09 19:23:16.096 UTC [27485] LOG: shutting down
2026-08-09 19:23:16.223 UTC [27478] LOG: database system is shut down
2026-08-09T19:23:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45163 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:23:16 dist-test-slave-1vmg krb5kdc[29736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:23:16 dist-test-slave-1vmg krb5kdc[29736](info): set up 2 sockets
Aug 09 19:23:16 dist-test-slave-1vmg krb5kdc[29736](info): commencing operation
krb5kdc: starting...
W20260809 19:23:17.451556 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:23:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:23:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:23:17.561575 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46071
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:46071,127.9.91.125:38993,127.9.91.124:39051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:17.642369 29759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:17.642561 29759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:17.642598 29759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:17.643738 29759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:17.643777 29759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:17.643810 29759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:17.643827 29759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:17.643847 29759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:17.645273 29759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:46071,127.9.91.125:38993,127.9.91.124:39051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:17.645504 29759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:17.645709 29759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:17.650183 29766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:17.650207 29770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:17.650372 29759 server_base.cc:1034] running on GCE node
W20260809 19:23:17.650399 29765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:17.650645 29759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:17.650938 29759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:17.652076 29759 hybrid_clock.cc:648] HybridClock initialized: now 1786303397652068 us; error 29 us; skew 500 ppm
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:17.658406 29759 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:17.658820 29759 webserver.cc:460] Webserver started at http://127.9.91.126:35387/ using document root <none> and password file <none>
I20260809 19:23:17.659041 29759 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:17.659086 29759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:17.659204 29759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:17.660230 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "4323051464724408aee2728aee5a14bc"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "751d32e9821f9706541fae14136c534a"
server_key_iv: "45b997e7c3529fc0f468d4d51146573f"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.660557 29759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "4323051464724408aee2728aee5a14bc"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "751d32e9821f9706541fae14136c534a"
server_key_iv: "45b997e7c3529fc0f468d4d51146573f"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.661759 29759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:23:17.662379 29781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:17.662508 29759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:17.662593 29759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "4323051464724408aee2728aee5a14bc"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "751d32e9821f9706541fae14136c534a"
server_key_iv: "45b997e7c3529fc0f468d4d51146573f"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.662653 29759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:17.670744 29759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:17.672741 29759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:17.672868 29759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:17.676783 29759 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:46071
I20260809 19:23:17.676817 29864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:46071 every 8 connection(s)
I20260809 19:23:17.677124 29759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:17.677927 29866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:17.679399 29866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:17.685582 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 29759
I20260809 19:23:17.685691 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:23:17.685911 9581 external_mini_cluster.cc:1428] Setting key 5f3718c3a835bd2c7e35843e39467960
I20260809 19:23:17.688180 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.679989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46071 (local address 127.0.0.1:43944)
0809 19:23:17.680162 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.680205 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:17.680668 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:17.680909 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:17.680912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:17.681161 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:17.681302 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.681306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.682257 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.682258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.682931 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.682934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.683375 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.683376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.683442 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.683822 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:17.683830 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:17.685081 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:17.686637 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.686645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.686647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.686884 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.687138 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.687139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.687139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.687244 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.687581 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:17.687589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:17.687684 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 19:23:17.687777 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 19:23:17.687943 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260809 19:23:17.688211 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.680015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.9.91.126:46071)
0809 19:23:17.680177 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.680186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.680289 (+ 103us) server_negotiation.cc:407] Connection header received
0809 19:23:17.680805 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.680807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.680823 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.680896 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.681389 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.682173 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.683001 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.683305 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.683398 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.685161 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.685168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.685168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.685174 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.686555 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.686995 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.686997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.686997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.687082 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.687298 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.687299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.687299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.687496 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.687582 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.687858 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.688079 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260809 19:23:17.688957 29866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:23:17.689252 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.689144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58674)
0809 19:23:17.689185 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.689231 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 19:23:17.689397 29784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
I20260809 19:23:17.689795 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.689708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58678)
0809 19:23:17.689749 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.689777 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 19:23:17.689893 29866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:38993: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.91.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab.
Entry for principal kudu/127.9.91.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab.
I20260809 19:23:17.709671 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:38993
--webserver_interface=127.9.91.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab
--principal=kudu/127.9.91.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:46071,127.9.91.125:38993,127.9.91.124:39051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:17.739122 29866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } attempt: 1
I20260809 19:23:17.739642 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.739495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58680)
0809 19:23:17.739559 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.739607 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 19:23:17.739976 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.739884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58688)
0809 19:23:17.739925 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.739953 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 19:23:17.740085 29866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:38993: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
W20260809 19:23:17.790113 29878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:17.790297 29878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:17.790318 29878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:17.791424 29878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:17.791460 29878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:17.791473 29878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:17.791496 29878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:17.791519 29878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:17.792949 29878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:46071,127.9.91.125:38993,127.9.91.124:39051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab
--principal=kudu/127.9.91.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:38993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.91.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:17.793154 29878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:17.793357 29878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:17.797757 29884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:17.797775 29888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:17.797772 29885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:17.798247 29878 server_base.cc:1034] running on GCE node
I20260809 19:23:17.798501 29878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:17.798763 29878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:17.799901 29878 hybrid_clock.cc:648] HybridClock initialized: now 1786303397799871 us; error 43 us; skew 500 ppm
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:17.802191 29878 init.cc:377] Logged in from keytab as kudu/127.9.91.125@KRBTEST.COM (short username kudu)
I20260809 19:23:17.802570 29878 webserver.cc:460] Webserver started at http://127.9.91.125:40669/ using document root <none> and password file <none>
I20260809 19:23:17.802778 29878 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:17.802825 29878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:17.802973 29878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:17.803974 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data/instance:
uuid: "73b36dc1499641eea6358487e442036d"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "fb4410730bf425ac91935cf9fb650220"
server_key_iv: "922bde8c725cfcff21e266eecc98f08d"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.804292 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/wal/instance:
uuid: "73b36dc1499641eea6358487e442036d"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "fb4410730bf425ac91935cf9fb650220"
server_key_iv: "922bde8c725cfcff21e266eecc98f08d"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.805526 29878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:23:17.806131 29897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:17.806273 29878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:17.806355 29878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/wal
uuid: "73b36dc1499641eea6358487e442036d"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "fb4410730bf425ac91935cf9fb650220"
server_key_iv: "922bde8c725cfcff21e266eecc98f08d"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.806411 29878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:17.808241 29866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } attempt: 2
I20260809 19:23:17.808598 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.808483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58698)
0809 19:23:17.808533 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.808562 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:23:17.808926 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.808857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58706)
0809 19:23:17.808890 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.808911 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:23:17.809000 29866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:38993: Network error: Client connection negotiation failed: client connection to 127.9.91.125:38993: connect: Connection refused (error 111)
I20260809 19:23:17.820783 29878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:17.822314 29878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:17.822433 29878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:17.826212 29878 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.125:38993
I20260809 19:23:17.826244 29980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.125:38993 every 8 connection(s)
I20260809 19:23:17.826520 29878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
I20260809 19:23:17.827339 29981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:17.828703 29981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:17.834160 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 29878
I20260809 19:23:17.834285 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-1/wal/instance
I20260809 19:23:17.834484 9581 external_mini_cluster.cc:1428] Setting key d16e3a5921de0f86bbb976d3d14f280a
I20260809 19:23:17.837177 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.829278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.9.91.126:46071)
0809 19:23:17.829327 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.829329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.829489 (+ 160us) server_negotiation.cc:407] Connection header received
0809 19:23:17.829951 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.829953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.829967 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.830018 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.830692 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.831235 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.832222 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.832435 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.832500 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.834449 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.834451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.834452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.834460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.835918 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.836299 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.836301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.836302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.836361 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.836573 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.836575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.836575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.836694 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.836750 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.836988 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.837089 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:23:17.837304 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.829248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46071 (local address 127.0.0.1:43956)
0809 19:23:17.829382 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.829415 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:17.829852 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:17.830081 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:17.830084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:17.830319 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:17.830496 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.830499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.831327 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.831328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.832127 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.832130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.832522 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.832524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.832636 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.833029 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:17.833037 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:17.834355 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:17.836000 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.836006 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.836008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.836224 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.836417 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.836418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.836419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.836515 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.836749 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:17.836756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:17.836875 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 19:23:17.836957 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:23:17.837107 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260809 19:23:17.837846 29981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.91.124@KRBTEST.COM; defaulting to no policy
I20260809 19:23:17.844928 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.838085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58712)
0809 19:23:17.838304 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.838315 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:17.838370 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:17.838508 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:17.838509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:17.838652 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:17.838708 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.838710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.839675 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.839676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.840411 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.840414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.840852 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.840853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.840925 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.841257 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:17.841262 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:17.842510 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:17.844062 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.844063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.844064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.844225 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.844389 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.844390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.844390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.844446 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.844709 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:17.844709 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:17.844743 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 19:23:17.844793 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 19:23:17.844870 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260809 19:23:17.845042 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.838160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.9.91.125:38993)
0809 19:23:17.838304 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.838307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.838363 (+ 56us) server_negotiation.cc:407] Connection header received
0809 19:23:17.838426 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.838427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.838438 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.838492 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.838807 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.839582 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.840499 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.840785 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.840835 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.842608 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.842615 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.842616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.842621 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.843990 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.844287 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.844289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.844290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.844337 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.844499 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.844500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.844500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.844656 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.844717 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.844814 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.844923 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260809 19:23:17.845549 29981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:23:17.845849 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.845751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56580)
0809 19:23:17.845789 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.845831 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 19:23:17.846021 29898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.124@KRBTEST.COM" created.
I20260809 19:23:17.846432 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.846346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56588)
0809 19:23:17.846382 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.846410 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:23:17.846541 29981 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:39051: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab.
Entry for principal kudu/127.9.91.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab.
I20260809 19:23:17.858855 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:39051
--webserver_interface=127.9.91.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab
--principal=kudu/127.9.91.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:46071,127.9.91.125:38993,127.9.91.124:39051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:17.895737 29981 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } attempt: 1
I20260809 19:23:17.896185 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.896066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56600)
0809 19:23:17.896120 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.896154 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 19:23:17.896513 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.896424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56616)
0809 19:23:17.896461 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.896488 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:23:17.896597 29981 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:39051: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
I20260809 19:23:17.900133 29866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } attempt: 3
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:23:17.907075 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.900422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58724)
0809 19:23:17.900483 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.900496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:17.900556 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:17.900707 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:17.900708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:17.900889 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:17.900966 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.900968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.901636 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.901637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.902431 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.902434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.902797 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.902797 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.902874 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.903305 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:17.903311 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:17.904637 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:17.906067 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.906068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.906068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.906245 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.906431 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.906431 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.906432 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.906503 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.906734 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:17.906735 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:17.906786 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 19:23:17.906855 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:23:17.906976 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:23:17.907087 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.900496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.9.91.125:38993)
0809 19:23:17.900545 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.900547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.900559 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:23:17.900626 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.900627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.900638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.900690 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.901043 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.901553 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.902510 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.902733 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.902791 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.904717 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.904719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.904719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.904727 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.905969 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.906312 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.906313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.906313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.906370 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.906561 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.906562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.906562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.906671 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.906734 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.906885 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.906987 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:23:17.907539 29866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:23:17.907851 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.907757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56624)
0809 19:23:17.907801 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.907831 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:23:17.908164 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.908075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56634)
0809 19:23:17.908115 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.908140 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 19:23:17.908245 29866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:39051: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
I20260809 19:23:17.939370 29866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } attempt: 1
I20260809 19:23:17.939706 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.939608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56640)
0809 19:23:17.939657 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.939683 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:23:17.939646 29988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:17.939860 29988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:17.939903 29988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 19:23:17.940064 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.939967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56642)
0809 19:23:17.940005 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.940044 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:23:17.940153 29866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:39051: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
W20260809 19:23:17.941061 29988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:17.941107 29988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:17.941126 29988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:17.941144 29988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:17.941167 29988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:17.942564 29988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:46071,127.9.91.125:38993,127.9.91.124:39051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab
--principal=kudu/127.9.91.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:39051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.91.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:17.942791 29988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:17.943019 29988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:17.947206 29997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:17.947248 29993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:17.947288 29995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:17.947484 29988 server_base.cc:1034] running on GCE node
I20260809 19:23:17.947711 29988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:17.947962 29988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:17.949097 29988 hybrid_clock.cc:648] HybridClock initialized: now 1786303397949081 us; error 27 us; skew 500 ppm
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:17.951345 29988 init.cc:377] Logged in from keytab as kudu/127.9.91.124@KRBTEST.COM (short username kudu)
I20260809 19:23:17.951736 29988 webserver.cc:460] Webserver started at http://127.9.91.124:39569/ using document root <none> and password file <none>
I20260809 19:23:17.951956 29988 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:17.952013 29988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:17.952149 29988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:17.953229 29988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data/instance:
uuid: "f3e30729d0b44426ab90cb66cfead3a3"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "673760d87a06d5d260ff4e2387806e56"
server_key_iv: "a9897633c9f56939c8af69f1fc4e7c3f"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.953542 29988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/wal/instance:
uuid: "f3e30729d0b44426ab90cb66cfead3a3"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "673760d87a06d5d260ff4e2387806e56"
server_key_iv: "a9897633c9f56939c8af69f1fc4e7c3f"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.954720 29988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:23:17.955395 30006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:17.955524 29988 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:17.955605 29988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/wal
uuid: "f3e30729d0b44426ab90cb66cfead3a3"
format_stamp: "Formatted at 2026-08-09 19:23:17 on dist-test-slave-1vmg"
server_key: "673760d87a06d5d260ff4e2387806e56"
server_key_iv: "a9897633c9f56939c8af69f1fc4e7c3f"
server_key_version: "encryptionkey@0"
I20260809 19:23:17.955668 29988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:17.964725 29981 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } attempt: 2
I20260809 19:23:17.965065 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.964949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56644)
0809 19:23:17.964998 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.965023 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:23:17.965411 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.965308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56658)
0809 19:23:17.965344 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.965387 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:23:17.965504 29981 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:39051: Network error: Client connection negotiation failed: client connection to 127.9.91.124:39051: connect: Connection refused (error 111)
I20260809 19:23:17.969841 29988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:17.971470 29988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:17.971582 29988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:17.975369 29988 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.124:39051
I20260809 19:23:17.975397 30087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.124:39051 every 8 connection(s)
I20260809 19:23:17.975687 29988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
I20260809 19:23:17.976454 30088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:17.977787 30088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:17.983011 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 29988
I20260809 19:23:17.983140 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786303248613589-9581-0/minicluster-data/master-2/wal/instance
I20260809 19:23:17.983311 9581 external_mini_cluster.cc:1428] Setting key 4d1d4af2502cfff84ad56409adaa447c
I20260809 19:23:17.985641 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.978371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.9.91.126:46071)
0809 19:23:17.978418 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.978420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.978601 (+ 181us) server_negotiation.cc:407] Connection header received
0809 19:23:17.979075 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.979077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.979099 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.979146 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.979720 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.980209 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.981123 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.981308 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.981363 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.983201 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.983203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.983203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.983210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.984450 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.984813 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.984814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.984815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.984867 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.985075 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.985076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.985076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.985186 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.985243 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.985483 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.985561 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:23:17.985798 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.978340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:46071 (local address 127.0.0.1:43970)
0809 19:23:17.978499 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.978530 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:17.978987 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:17.979231 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:17.979235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:17.979452 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:17.979636 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.979640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.980289 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.980290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.981037 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.981039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.981376 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.981377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.981491 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.981865 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:17.981872 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:17.983112 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:17.984525 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.984532 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.984534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.984747 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.984922 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.984923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.984923 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.985021 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.985241 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:17.985248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:17.985372 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 19:23:17.985455 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:23:17.985606 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260809 19:23:17.986297 30088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:23:17.991518 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.983860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56672 (local address 127.9.91.124:39051)
0809 19:23:17.984090 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.984094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.984117 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:23:17.984138 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.984140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.984152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.984216 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.984767 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.985416 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.986249 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.986700 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.986755 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.988900 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.988907 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.988907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.988913 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.990404 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.990754 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.990756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.990757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.990805 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.991000 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.991001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.991001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.991115 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.991211 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.991319 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.991422 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":72,"threads_started":1}
I20260809 19:23:17.992233 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.986483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:38993 (local address 127.0.0.1:58728)
0809 19:23:17.986520 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.986529 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:17.986562 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:17.986681 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:17.986682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:17.986781 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:17.986822 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.986823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.987426 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.987427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.987910 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.987912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.988227 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.988228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.988297 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.988624 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:17.988629 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:17.989839 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:17.991300 (+ 1461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.991301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.991301 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.991522 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.991723 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.991724 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.991724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.991773 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.991994 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:17.991995 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:17.992034 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 19:23:17.992093 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:23:17.992173 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 19:23:17.992285 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.986539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.9.91.125:38993)
0809 19:23:17.986578 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.986580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.986587 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:23:17.986609 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.986610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.986619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.986679 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.986890 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.987362 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.987977 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.988163 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.988213 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.989918 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.989919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.989920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.989926 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.991229 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.991582 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.991584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.991584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.991658 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.991827 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.991828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.991828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.991939 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.991996 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.992128 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.992206 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:23:17.992676 30088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
Aug 09 19:23:17 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:17.998369 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.992872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56680)
0809 19:23:17.992909 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:17.992917 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:17.992948 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:17.993041 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:17.993042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:17.993155 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:17.993176 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.993177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.993731 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.993732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.994174 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:17.994176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.994481 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.994482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:17.994525 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.994865 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:17.994869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:17.996011 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:17.997433 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.997434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.997434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.997653 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.997839 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:17.997839 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:17.997840 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:17.997907 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:17.998101 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:17.998102 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:17.998147 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 19:23:17.998222 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:23:17.998315 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:23:17.998471 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.992924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56680 (local address 127.9.91.124:39051)
0809 19:23:17.992954 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.992955 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.992963 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:17.992990 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.992991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.992999 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.993037 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.993232 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.993671 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.994234 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.994430 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.994478 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.996087 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.996088 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.996088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.996093 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.997337 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.997727 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.997728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.997728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.997776 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.997967 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.997967 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.997968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.998047 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.998100 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.998239 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.998368 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":16}
I20260809 19:23:17.999415 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:17.992331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43982 (local address 127.9.91.126:46071)
0809 19:23:17.992508 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:17.992509 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:17.992517 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:17.992533 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:17.992534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:17.992542 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:17.992585 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:17.993100 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.993519 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.994124 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:17.994308 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:17.994357 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:17.996614 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:17.996615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:17.996615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:17.996621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:17.998195 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.998585 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.998586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.998587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.998655 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:17.998884 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:17.998885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:17.998885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:17.999031 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:17.999220 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:17.999258 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 19:23:17.999331 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260809 19:23:17.999781 30088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3: Bootstrap starting.
I20260809 19:23:18.000648 30088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:18.000914 30088 log.cc:824] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:18.001645 30088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3: No bootstrap required, opened a new log
I20260809 19:23:18.003321 30088 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.003427 30088 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:18.003446 30088 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e30729d0b44426ab90cb66cfead3a3, State: Initialized, Role: FOLLOWER
I20260809 19:23:18.003546 30088 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.003816 30088 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:18.003803 30098 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.003914 30098 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 19:23:18 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
W20260809 19:23:18.005590 30112 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:23:18.005647 30112 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:23:18.007732 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.000308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58730 (local address 127.9.91.125:38993)
0809 19:23:18.000348 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:18.000350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:18.000359 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:23:18.000380 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:18.000381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:18.000388 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:18.000446 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:18.000908 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.001465 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.002115 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.002383 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.002455 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.004685 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:18.004687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:18.004687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:18.004693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:18.006681 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.007102 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.007103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.007103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.007148 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.007344 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.007344 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.007344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.007423 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:18.007537 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:18.007624 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 19:23:18.007679 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:23:18.005812 29988 master_runner.cc:186] Error getting master registration for 127.9.91.126:46071: OK, instance_id { permanent_uuid: "4323051464724408aee2728aee5a14bc" instance_seqno: 1786303397675693 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:23:18.008446 29988 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:23:18.009052 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:23:18.015280 29866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } attempt: 2
I20260809 19:23:18.018692 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.010664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.9.91.125:38993)
0809 19:23:18.010700 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:18.010702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:18.010709 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:23:18.010726 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:18.010726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:18.010734 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:18.010777 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:18.011380 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.012047 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.013132 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.013405 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.013439 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.014985 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:18.014987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:18.014987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:18.014993 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:18.016754 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.017797 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.017798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.017799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.017854 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.018047 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.018047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.018048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.018160 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:18.018251 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:18.018535 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 19:23:18.018612 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:23:18.019475 30093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.010698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56690 (local address 127.9.91.124:39051)
0809 19:23:18.010736 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:18.010737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:18.011089 (+ 352us) server_negotiation.cc:407] Connection header received
0809 19:23:18.011194 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:18.011195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:18.011204 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:18.011234 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:18.011987 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.012613 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.013578 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.014265 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.014314 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.016236 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:18.016238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:18.016238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:18.016245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:18.017842 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.018263 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.018264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.018264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.018326 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.018779 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.018780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.018780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.018973 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:18.019041 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:18.019314 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 19:23:18.019393 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:23:18.020184 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.010282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.9.91.126:46071)
0809 19:23:18.010315 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:18.010316 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:18.010436 (+ 120us) server_negotiation.cc:407] Connection header received
0809 19:23:18.010525 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:18.010526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:18.010535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:18.010573 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:18.011358 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.012047 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.013124 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.013402 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.013439 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.014999 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:18.015000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:18.015000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:18.015006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:18.017435 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.019355 (+ 1920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.019356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.019356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.019411 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.019641 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.019642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.019642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.019772 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:18.019843 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:18.020039 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 19:23:18.020103 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 19:23:18 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:23:18.023764 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.015491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56706)
0809 19:23:18.015583 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:18.015591 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:18.015627 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:18.016112 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:18.016113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:18.016456 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:18.016709 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:18.016712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.017930 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.017931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:18.018622 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:18.018625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.019308 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.019309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:18.019363 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.019924 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:18.019929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:18.021353 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:18.022795 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:18.022796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:18.022796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:18.023050 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:18.023235 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:18.023235 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:18.023236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:18.023297 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:18.023480 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:18.023481 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:18.023529 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:23:18.023589 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:23:18.023672 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 19:23:18.024243 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.015744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.9.91.124:39051)
0809 19:23:18.015964 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:18.015966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:18.015974 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:18.015992 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:18.015993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:18.016002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:18.016785 (+ 783us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:18.016844 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.017844 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.018710 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.019216 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.020615 (+ 1399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.021431 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:18.021432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:18.021433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:18.021438 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:18.022723 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.023123 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.023124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.023124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.023184 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.023350 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.023351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.023351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.023431 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:18.023483 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:18.024053 (+ 570us) server_negotiation.cc:299] Negotiation successful
0809 19:23:18.024134 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
I20260809 19:23:18.025396 29866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: Bootstrap starting.
I20260809 19:23:18.025941 29866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:18.026188 29866 log.cc:824] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:18.027100 29866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: No bootstrap required, opened a new log
I20260809 19:23:18.028429 29866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.028527 29866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:18.028559 29866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4323051464724408aee2728aee5a14bc, State: Initialized, Role: FOLLOWER
I20260809 19:23:18.028676 29866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.028901 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.028944 29866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:18.028988 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:23:18.030548 30142 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:23:18.030601 30142 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:23:18.031248 29759 master_runner.cc:186] Error getting master registration for 127.9.91.125:38993: OK, instance_id { permanent_uuid: "73b36dc1499641eea6358487e442036d" instance_seqno: 1786303397825211 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:23:18.034202 29759 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:23:18.056654 29981 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } attempt: 3
Aug 09 19:23:18 dist-test-slave-1vmg krb5kdc[29736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303397, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:23:18.062765 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.056977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.9.91.124:39051)
0809 19:23:18.057010 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:18.057011 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:18.057018 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:23:18.057036 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:18.057037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:18.057045 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:18.057089 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:18.057328 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.057820 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.058493 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.058735 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.058791 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.060528 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:18.060529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:18.060529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:18.060536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:18.061797 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.062092 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.062093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.062093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.062145 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:18.062304 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:18.062305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:18.062305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:18.062417 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:18.062473 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:18.062613 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 19:23:18.062679 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:23:18.062768 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:18.056880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:39051 (local address 127.0.0.1:56714)
0809 19:23:18.056926 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:18.056936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:18.056974 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:18.057100 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:18.057101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:18.057232 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:18.057267 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:18.057268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.057890 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.057891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:18.058408 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:18.058410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:18.058801 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:18.058802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:18.058855 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:18.059246 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:18.059251 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:18.060456 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:18.061870 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:18.061871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:18.061872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:18.062032 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:18.062198 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:18.062199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:18.062199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:18.062256 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:18.062473 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:18.062474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:18.062526 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 19:23:18.062585 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 19:23:18.062679 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:23:18.064188 29981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: Bootstrap starting.
I20260809 19:23:18.064787 29981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:18.065068 29981 log.cc:824] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:18.065754 29981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: No bootstrap required, opened a new log
I20260809 19:23:18.067239 29981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.067341 29981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:18.067373 29981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b36dc1499641eea6358487e442036d, State: Initialized, Role: FOLLOWER
I20260809 19:23:18.067488 29981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.067731 30147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.067785 29981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:18.067862 30147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:23:18.069461 30164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:23:18.069574 30164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:23:18.071497 29878 master_runner.cc:263] No leader master found from master 73b36dc1499641eea6358487e442036d
I20260809 19:23:18.071550 29878 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 19:23:18.072938 29906 tablet.cc:2369] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:23:18.173543 29792 tablet.cc:2369] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:23:18.222045 30018 tablet.cc:2369] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:23:18.314222 30098 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:18.314354 30098 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.314597 30098 leader_election.cc:288] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4323051464724408aee2728aee5a14bc (127.9.91.126:46071), 73b36dc1499641eea6358487e442036d (127.9.91.125:38993)
I20260809 19:23:18.314812 29823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3e30729d0b44426ab90cb66cfead3a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4323051464724408aee2728aee5a14bc" is_pre_election: true
I20260809 19:23:18.314848 29937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3e30729d0b44426ab90cb66cfead3a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b36dc1499641eea6358487e442036d" is_pre_election: true
I20260809 19:23:18.314973 29823 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e30729d0b44426ab90cb66cfead3a3 in term 0.
I20260809 19:23:18.315016 29937 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3e30729d0b44426ab90cb66cfead3a3 in term 0.
I20260809 19:23:18.315172 30011 leader_election.cc:302] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4323051464724408aee2728aee5a14bc, f3e30729d0b44426ab90cb66cfead3a3; no voters:
I20260809 19:23:18.315310 30098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:23:18.315378 30098 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:23:18.315398 30098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:18.316119 30098 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.316243 30098 leader_election.cc:288] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [CANDIDATE]: Term 1 election: Requested vote from peers 4323051464724408aee2728aee5a14bc (127.9.91.126:46071), 73b36dc1499641eea6358487e442036d (127.9.91.125:38993)
I20260809 19:23:18.316376 29823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3e30729d0b44426ab90cb66cfead3a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4323051464724408aee2728aee5a14bc"
I20260809 19:23:18.316450 29823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:18.316429 29937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3e30729d0b44426ab90cb66cfead3a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b36dc1499641eea6358487e442036d"
I20260809 19:23:18.316519 29937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:18.317153 29823 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e30729d0b44426ab90cb66cfead3a3 in term 1.
I20260809 19:23:18.317219 29937 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3e30729d0b44426ab90cb66cfead3a3 in term 1.
I20260809 19:23:18.317332 30011 leader_election.cc:302] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4323051464724408aee2728aee5a14bc, f3e30729d0b44426ab90cb66cfead3a3; no voters:
I20260809 19:23:18.317437 30098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:18.317569 30098 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [term 1 LEADER]: Becoming Leader. State: Replica: f3e30729d0b44426ab90cb66cfead3a3, State: Running, Role: LEADER
I20260809 19:23:18.317674 30098 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } }
I20260809 19:23:18.318053 30167 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e30729d0b44426ab90cb66cfead3a3. Latest consensus state: current_term: 1 leader_uuid: "f3e30729d0b44426ab90cb66cfead3a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.318130 30167 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:18.318336 30171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:23:18.318524 30171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:23:18.319860 29823 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [term 1 FOLLOWER]: Refusing update from remote peer f3e30729d0b44426ab90cb66cfead3a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:18.320111 29937 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [term 1 FOLLOWER]: Refusing update from remote peer f3e30729d0b44426ab90cb66cfead3a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:18.320132 30167 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:18.320343 30167 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:18.321341 30147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e30729d0b44426ab90cb66cfead3a3. Latest consensus state: current_term: 1 leader_uuid: "f3e30729d0b44426ab90cb66cfead3a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.321341 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e30729d0b44426ab90cb66cfead3a3. Latest consensus state: current_term: 1 leader_uuid: "f3e30729d0b44426ab90cb66cfead3a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.321429 30147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:23:18.321429 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:23:18.322247 30098 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3e30729d0b44426ab90cb66cfead3a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.322368 30175 mvcc.cc:205] Tried to move back new op lower bound from 7316698719516041216 to 7316698719509667840. Current Snapshot: MvccSnapshot[applied={T|T < 7316698719516041216}]
I20260809 19:23:18.322341 30179 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3e30729d0b44426ab90cb66cfead3a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.322330 30098 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:18.322438 30179 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:18.322474 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3e30729d0b44426ab90cb66cfead3a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.322530 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:23:18.322522 30147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3e30729d0b44426ab90cb66cfead3a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4323051464724408aee2728aee5a14bc" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46071 } } peers { permanent_uuid: "73b36dc1499641eea6358487e442036d" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 38993 } } peers { permanent_uuid: "f3e30729d0b44426ab90cb66cfead3a3" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 39051 } } }
I20260809 19:23:18.322597 30147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:23:18.322721 30173 mvcc.cc:205] Tried to move back new op lower bound from 7316698719516041216 to 7316698719509667840. Current Snapshot: MvccSnapshot[applied={T|T < 7316698719516041216}]
I20260809 19:23:18.322824 30171 catalog_manager.cc:1347] Generated new cluster ID: ac7d164f4ee944a082993ed7e26c041b
I20260809 19:23:18.322871 30171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:23:18.330080 30171 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:23:18.330693 30171 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:23:18.339592 30171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3e30729d0b44426ab90cb66cfead3a3: Generated new TSK 0
I20260809 19:23:18.339775 30171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:23:19.031023 30142 catalog_manager.cc:1259] Loaded cluster ID: ac7d164f4ee944a082993ed7e26c041b
I20260809 19:23:19.031090 30142 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: loading cluster ID for follower catalog manager: success
I20260809 19:23:19.031881 30142 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: acquiring CA information for follower catalog manager: success
I20260809 19:23:19.032187 30142 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4323051464724408aee2728aee5a14bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:23:19.069983 30164 catalog_manager.cc:1259] Loaded cluster ID: ac7d164f4ee944a082993ed7e26c041b
I20260809 19:23:19.070039 30164 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: loading cluster ID for follower catalog manager: success
I20260809 19:23:19.070817 30164 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: acquiring CA information for follower catalog manager: success
I20260809 19:23:19.071120 30164 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73b36dc1499641eea6358487e442036d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:23:19.408944 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 29759
I20260809 19:23:19.418296 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 29878
I20260809 19:23:19.429725 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 29988
2026-08-09T19:23:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3114 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:23:19 dist-test-slave-1vmg krb5kdc[30194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:23:19 dist-test-slave-1vmg krb5kdc[30194](info): set up 2 sockets
Aug 09 19:23:19 dist-test-slave-1vmg krb5kdc[30194](info): commencing operation
krb5kdc: starting...
W20260809 19:23:20.567368 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:23:20 dist-test-slave-1vmg krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:23:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:23:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:23:20.677241 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38819
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:38819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:20.757686 30211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:20.757880 30211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:20.757901 30211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:20.759038 30211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:20.759081 30211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:20.759094 30211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:20.759107 30211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:20.759130 30211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:20.760598 30211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:38819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:20.760813 30211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:20.761027 30211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:20.765658 30222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:20.765689 30217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:20.765666 30218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:20.765725 30211 server_base.cc:1034] running on GCE node
I20260809 19:23:20.766081 30211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:20.766333 30211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:20.767481 30211 hybrid_clock.cc:648] HybridClock initialized: now 1786303400767469 us; error 29 us; skew 500 ppm
Aug 09 19:23:20 dist-test-slave-1vmg krb5kdc[30194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303400, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:20.769980 30211 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:20.770455 30211 webserver.cc:460] Webserver started at http://127.9.91.126:33815/ using document root <none> and password file <none>
I20260809 19:23:20.770640 30211 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:20.770680 30211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:20.770771 30211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:20.771807 30211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "350cf6d323fa4638b01a6ed5c4aa1564"
format_stamp: "Formatted at 2026-08-09 19:23:20 on dist-test-slave-1vmg"
server_key: "850ea8323ab32185efc6714f29302427"
server_key_iv: "a5cd3ba213325f07cdf4c7543e808b7f"
server_key_version: "encryptionkey@0"
I20260809 19:23:20.772140 30211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "350cf6d323fa4638b01a6ed5c4aa1564"
format_stamp: "Formatted at 2026-08-09 19:23:20 on dist-test-slave-1vmg"
server_key: "850ea8323ab32185efc6714f29302427"
server_key_iv: "a5cd3ba213325f07cdf4c7543e808b7f"
server_key_version: "encryptionkey@0"
I20260809 19:23:20.773412 30211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:23:20.774137 30231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:20.774312 30211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:20.774399 30211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "350cf6d323fa4638b01a6ed5c4aa1564"
format_stamp: "Formatted at 2026-08-09 19:23:20 on dist-test-slave-1vmg"
server_key: "850ea8323ab32185efc6714f29302427"
server_key_iv: "a5cd3ba213325f07cdf4c7543e808b7f"
server_key_version: "encryptionkey@0"
I20260809 19:23:20.774456 30211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:20.792223 30211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:20.794194 30211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:20.794322 30211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:20.798189 30211 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:38819
I20260809 19:23:20.798228 30313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:38819 every 8 connection(s)
I20260809 19:23:20.798542 30211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:20.799396 30314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:20.801271 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 30211
I20260809 19:23:20.801362 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:23:20.801544 9581 external_mini_cluster.cc:1428] Setting key af24821810990bafc5ec5b65031a0e0d
I20260809 19:23:20.801786 30314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564: Bootstrap starting.
I20260809 19:23:20.802630 30314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:20.802983 30314 log.cc:824] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:20.803736 30314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564: No bootstrap required, opened a new log
I20260809 19:23:20.805851 30314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38819 } }
I20260809 19:23:20.806025 30314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:20.806058 30314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350cf6d323fa4638b01a6ed5c4aa1564, State: Initialized, Role: FOLLOWER
I20260809 19:23:20.806190 30314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38819 } }
I20260809 19:23:20.806255 30314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:23:20.806281 30314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:23:20.806329 30314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:20.807026 30314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38819 } }
I20260809 19:23:20.807147 30314 leader_election.cc:302] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 350cf6d323fa4638b01a6ed5c4aa1564; no voters:
I20260809 19:23:20.807314 30314 leader_election.cc:288] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:23:20.807346 30319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:20.807511 30319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [term 1 LEADER]: Becoming Leader. State: Replica: 350cf6d323fa4638b01a6ed5c4aa1564, State: Running, Role: LEADER
I20260809 19:23:20.807581 30314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:20.807669 30319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38819 } }
I20260809 19:23:20.807974 30319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 350cf6d323fa4638b01a6ed5c4aa1564. Latest consensus state: current_term: 1 leader_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38819 } } }
I20260809 19:23:20.808193 30319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:20.808130 30323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "350cf6d323fa4638b01a6ed5c4aa1564" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38819 } } }
I20260809 19:23:20.808560 30323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:20.808692 30331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 19:23:20 dist-test-slave-1vmg krb5kdc[30194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:20.809178 30331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:23:20.810856 30331 catalog_manager.cc:1347] Generated new cluster ID: 17167dee00dd4ad88415b5306257091b
I20260809 19:23:20.810938 30331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:23:20.812774 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:20.801961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41428 (local address 127.9.91.126:38819)
0809 19:23:20.802146 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:20.802156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:20.802182 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:23:20.802585 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:20.802591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:20.802635 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:20.802767 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:20.804077 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:20.804911 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:20.805965 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:20.806504 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:20.806582 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:20.809473 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:20.809477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:20.809477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:20.809489 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:20.811343 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:20.811696 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:20.811699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:20.811699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:20.811789 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:20.811975 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:20.811976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:20.811977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:20.812212 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:20.812393 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:20.812472 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:23:20.812606 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260809 19:23:20.815330 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:23:20.822182 30331 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:23:20.822829 30331 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:23:20.829475 30331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 350cf6d323fa4638b01a6ed5c4aa1564: Generated new TSK 0
I20260809 19:23:20.829631 30331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:23:20.868613 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 30211
2026-08-09T19:23:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1437 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:23:20 dist-test-slave-1vmg krb5kdc[30347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:23:20 dist-test-slave-1vmg krb5kdc[30347](info): set up 2 sockets
Aug 09 19:23:20 dist-test-slave-1vmg krb5kdc[30347](info): commencing operation
krb5kdc: starting...
W20260809 19:23:21.994782 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:23:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:23:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:23:22.117182 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36485
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:22.195551 30368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:22.195735 30368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:22.195765 30368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:22.196862 30368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:22.196902 30368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:22.196920 30368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:22.196936 30368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:22.196952 30368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:22.198400 30368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:22.198629 30368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:22.198827 30368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:22.203267 30382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.203341 30383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.203267 30387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:22.203580 30368 server_base.cc:1034] running on GCE node
I20260809 19:23:22.203835 30368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:22.204092 30368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:22.205225 30368 hybrid_clock.cc:648] HybridClock initialized: now 1786303402205208 us; error 34 us; skew 500 ppm
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:22.207530 30368 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:22.207903 30368 webserver.cc:460] Webserver started at http://127.9.91.126:41305/ using document root <none> and password file <none>
I20260809 19:23:22.208091 30368 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:22.208132 30368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:22.208218 30368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:22.209122 30368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "ed658dde74f6447daf5932837e4453a5"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "c67f24e9b4b226b577092fba17705758"
server_key_iv: "d897bb7d37bc5efbc14243111cd72ce3"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.209399 30368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "ed658dde74f6447daf5932837e4453a5"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "c67f24e9b4b226b577092fba17705758"
server_key_iv: "d897bb7d37bc5efbc14243111cd72ce3"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.210501 30368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:23:22.211179 30398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.211321 30368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:23:22.211404 30368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "ed658dde74f6447daf5932837e4453a5"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "c67f24e9b4b226b577092fba17705758"
server_key_iv: "d897bb7d37bc5efbc14243111cd72ce3"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.211463 30368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:22.220304 30368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:22.221830 30368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:22.221949 30368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:22.225629 30368 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:36485
I20260809 19:23:22.225666 30474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:36485 every 8 connection(s)
I20260809 19:23:22.225945 30368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:22.226809 30475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:22.228827 30475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Bootstrap starting.
I20260809 19:23:22.229337 30475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:22.229564 30475 log.cc:824] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:22.230170 30475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: No bootstrap required, opened a new log
I20260809 19:23:22.231062 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 30368
I20260809 19:23:22.231182 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:23:22.231354 9581 external_mini_cluster.cc:1428] Setting key ec550ec39e980c9f5d2305903d5a7d72
I20260809 19:23:22.231839 30475 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:22.232007 30475 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:22.232064 30475 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed658dde74f6447daf5932837e4453a5, State: Initialized, Role: FOLLOWER
I20260809 19:23:22.232182 30475 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:22.232249 30475 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:23:22.232277 30475 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:23:22.232365 30475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:22.233309 30475 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:22.233407 30475 leader_election.cc:302] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed658dde74f6447daf5932837e4453a5; no voters:
I20260809 19:23:22.233551 30475 leader_election.cc:288] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:23:22.233590 30483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:22.233731 30483 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 LEADER]: Becoming Leader. State: Replica: ed658dde74f6447daf5932837e4453a5, State: Running, Role: LEADER
I20260809 19:23:22.233798 30475 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:22.233888 30483 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:22.234187 30483 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed658dde74f6447daf5932837e4453a5. Latest consensus state: current_term: 1 leader_uuid: "ed658dde74f6447daf5932837e4453a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } } }
I20260809 19:23:22.234416 30483 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:22.234341 30485 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed658dde74f6447daf5932837e4453a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } } }
I20260809 19:23:22.234833 30485 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:22.234838 30494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:23:22.235427 30494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:23:22.236886 30494 catalog_manager.cc:1347] Generated new cluster ID: b0b9e61a09d5478e98562b1a8444780a
I20260809 19:23:22.236933 30494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:22.242441 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.231813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.9.91.126:36485)
0809 19:23:22.231958 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.231965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.231987 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:23:22.232084 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.232090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.232144 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.232267 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:22.233028 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.234127 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.235289 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.235860 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.235962 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.238586 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:22.238590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:22.238591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:22.238602 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:22.240451 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.240818 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.240821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.240822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.240954 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.241120 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.241121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.241121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.241409 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:22.242032 (+ 623us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.242114 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.242248 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260809 19:23:22.250873 30494 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:23:22.251567 30494 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:23:22.261195 30494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Generated new TSK 0
I20260809 19:23:22.261339 30494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:23:22.286903 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36485
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:22.366199 30515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:22.366394 30515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:22.366429 30515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:22.367645 30515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:22.367691 30515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:22.367728 30515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:23:22.369179 30515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:22.369417 30515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:22.369630 30515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:22.374292 30523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.374301 30522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:22.374523 30515 server_base.cc:1034] running on GCE node
W20260809 19:23:22.374305 30525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:22.374758 30515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:22.375016 30515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:22.376149 30515 hybrid_clock.cc:648] HybridClock initialized: now 1786303402376141 us; error 28 us; skew 500 ppm
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:22.378448 30515 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:23:22.378837 30515 webserver.cc:460] Webserver started at http://127.9.91.65:34573/ using document root <none> and password file <none>
I20260809 19:23:22.379055 30515 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:22.379103 30515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:22.379223 30515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:22.380218 30515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "4cb8ebfd206c4242a3a52b9cd1d63846"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "d383dc58078dd04081d40178675de485"
server_key_iv: "233ce102fbd80623bd4053e1f21c98ae"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.380529 30515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "4cb8ebfd206c4242a3a52b9cd1d63846"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "d383dc58078dd04081d40178675de485"
server_key_iv: "233ce102fbd80623bd4053e1f21c98ae"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.381769 30515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:23:22.382427 30536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.382580 30515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:22.382661 30515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "4cb8ebfd206c4242a3a52b9cd1d63846"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "d383dc58078dd04081d40178675de485"
server_key_iv: "233ce102fbd80623bd4053e1f21c98ae"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.382726 30515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:22.391398 30515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:22.393003 30515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:22.393131 30515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:22.393312 30515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:22.393584 30515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:22.393625 30515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.393651 30515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:22.393669 30515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.399402 30515 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:44001
I20260809 19:23:22.399475 30699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:44001 every 8 connection(s)
I20260809 19:23:22.399724 30515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:23:22.401232 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 30515
I20260809 19:23:22.401346 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:23:22.401563 9581 external_mini_cluster.cc:1428] Setting key f9a9f6722da7fa6aabfe2b524d77ceaf
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:22.408812 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.400356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:36495 (local address 127.9.91.126:36485)
0809 19:23:22.400429 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.400431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.400579 (+ 148us) server_negotiation.cc:407] Connection header received
0809 19:23:22.400975 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.400976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.400987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.401046 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:22.402086 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.402642 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.403985 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.404286 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.404319 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.406255 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:22.406256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:22.406257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:22.406264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:22.407583 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.407936 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.407938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.407938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.407990 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.408218 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.408219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.408220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.408324 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:22.408381 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.408637 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.408722 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:23:22.408933 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.400333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:36495)
0809 19:23:22.400470 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.400506 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.400892 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.401130 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.401134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.401829 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:22.402000 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.402004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.402737 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.402738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.403878 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.403881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.404392 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.404392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.404488 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.404910 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:22.404917 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:22.406168 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:22.407657 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:22.407664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:22.407666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:22.407869 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:22.408044 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:22.408045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:22.408045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:22.408166 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:22.408379 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:22.408386 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:22.408512 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.408604 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.408744 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":83,"threads_started":1}
I20260809 19:23:22.409359 30700 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36485
I20260809 19:23:22.409463 30700 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:22.409650 30700 heartbeater.cc:511] Master 127.9.91.126:36485 requested a full tablet report, sending...
I20260809 19:23:22.410145 30425 ts_manager.cc:194] Registered new tserver with Master: 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65:44001)
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
I20260809 19:23:22.410749 30425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:36495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:23:22.426679 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36485
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:22.504452 30706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:22.504627 30706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:22.504650 30706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:22.505734 30706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:22.505774 30706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:22.505800 30706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:23:22.507210 30706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:22.507404 30706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:22.507601 30706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:22.512257 30713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.512262 30711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.512327 30716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:22.512344 30706 server_base.cc:1034] running on GCE node
I20260809 19:23:22.512665 30706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:22.512874 30706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:22.514005 30706 hybrid_clock.cc:648] HybridClock initialized: now 1786303402513985 us; error 39 us; skew 500 ppm
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:22.516300 30706 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:23:22.516678 30706 webserver.cc:460] Webserver started at http://127.9.91.66:37435/ using document root <none> and password file <none>
I20260809 19:23:22.516858 30706 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:22.516906 30706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:22.517022 30706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:22.517936 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "e8372f39adb34f1dbb7e6dbac74f836e"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "9aaf1045dbd03cb6fc277770d83ee6c4"
server_key_iv: "fcc3a3bd4608729eec53426f2000c3e8"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.518244 30706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "e8372f39adb34f1dbb7e6dbac74f836e"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "9aaf1045dbd03cb6fc277770d83ee6c4"
server_key_iv: "fcc3a3bd4608729eec53426f2000c3e8"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.519484 30706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:23:22.520123 30725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.520264 30706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:22.520342 30706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "e8372f39adb34f1dbb7e6dbac74f836e"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "9aaf1045dbd03cb6fc277770d83ee6c4"
server_key_iv: "fcc3a3bd4608729eec53426f2000c3e8"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.520398 30706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:22.549132 30706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:22.550661 30706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:22.550781 30706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:22.551003 30706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:22.551271 30706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:22.551304 30706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.551330 30706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:22.551348 30706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.556661 30706 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:37103
I20260809 19:23:22.556733 30894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:37103 every 8 connection(s)
I20260809 19:23:22.556962 30706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:23:22.560959 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 30706
I20260809 19:23:22.561059 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:23:22.561270 9581 external_mini_cluster.cc:1428] Setting key b0853a6ff1fa169cd60d5d5af214ccee
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:22.565645 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.557558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:44711 (local address 127.9.91.126:36485)
0809 19:23:22.557601 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.557604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.557767 (+ 163us) server_negotiation.cc:407] Connection header received
0809 19:23:22.558206 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.558208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.558221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.558273 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:22.559155 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.559655 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.560595 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.560800 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.560856 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.563147 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:22.563148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:22.563148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:22.563155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:22.564466 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.564843 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.564844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.564845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.564905 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.565094 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.565095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.565095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.565218 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:22.565282 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.565475 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.565563 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:23:22.565696 30898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.557520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:44711)
0809 19:23:22.557662 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.557696 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.558116 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.558333 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.558337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.558842 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:22.559067 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.559072 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.559735 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.559736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.560507 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.560510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.560867 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.560868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.560988 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.561476 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:22.561485 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:22.563064 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:22.564557 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:22.564563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:22.564565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:22.564778 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:22.564961 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:22.564961 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:22.564961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:22.565044 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:22.565271 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:22.565273 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:22.565372 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.565443 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.565546 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260809 19:23:22.566345 30896 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36485
I20260809 19:23:22.566437 30896 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:22.566596 30896 heartbeater.cc:511] Master 127.9.91.126:36485 requested a full tablet report, sending...
I20260809 19:23:22.566951 30425 ts_manager.cc:194] Registered new tserver with Master: e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
I20260809 19:23:22.567317 30425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:44711
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:23:22.583627 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36485
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:22.661087 30903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:22.661260 30903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:22.661281 30903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:22.662374 30903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:22.662411 30903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:22.662437 30903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:23:22.663859 30903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:22.664091 30903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:22.664290 30903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:22.668946 30912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.668957 30911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.668946 30915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:22.669124 30903 server_base.cc:1034] running on GCE node
I20260809 19:23:22.669312 30903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:22.669528 30903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:22.670681 30903 hybrid_clock.cc:648] HybridClock initialized: now 1786303402670652 us; error 31 us; skew 500 ppm
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:22.675861 30903 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:23:22.676255 30903 webserver.cc:460] Webserver started at http://127.9.91.67:39901/ using document root <none> and password file <none>
I20260809 19:23:22.676419 30903 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:22.676455 30903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:22.676538 30903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:22.677415 30903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "44c51dde9b054c44a26708d22ac22655"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "f725af70fa71a3d29f254dae78dec019"
server_key_iv: "33097e4f12755f173d8d16cd3d72f44c"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.677690 30903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "44c51dde9b054c44a26708d22ac22655"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "f725af70fa71a3d29f254dae78dec019"
server_key_iv: "33097e4f12755f173d8d16cd3d72f44c"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.678790 30903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 19:23:22.679481 30924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.679620 30903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.679697 30903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "44c51dde9b054c44a26708d22ac22655"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "f725af70fa71a3d29f254dae78dec019"
server_key_iv: "33097e4f12755f173d8d16cd3d72f44c"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.679766 30903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:22.693392 30903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:22.694880 30903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:22.695034 30903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:22.695225 30903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:22.695482 30903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:22.695513 30903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.695537 30903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:22.695554 30903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.701212 30903 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:34487
I20260809 19:23:22.701247 31087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:34487 every 8 connection(s)
I20260809 19:23:22.701504 30903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:22.707442 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 30903
I20260809 19:23:22.707554 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:23:22.707712 9581 external_mini_cluster.cc:1428] Setting key dd0f855ad05b89f8b50f678452f4ea33
I20260809 19:23:22.709446 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.702101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:59229 (local address 127.9.91.126:36485)
0809 19:23:22.702150 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.702153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.702295 (+ 142us) server_negotiation.cc:407] Connection header received
0809 19:23:22.702693 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.702694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.702702 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.702749 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:22.703562 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.704044 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.704912 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.705084 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.705124 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.706964 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:22.706965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:22.706965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:22.706973 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:22.708311 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.708729 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.708730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.708730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.708783 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.708984 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.708985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.708985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.709103 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:22.709157 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.709328 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.709386 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:23:22.709590 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.702052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:59229)
0809 19:23:22.702192 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.702229 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.702624 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.702795 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.702798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.703321 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:22.703492 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.703496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.704105 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.704106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.704853 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.704854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.705168 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.705169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.705276 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.705625 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:22.705633 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:22.706852 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:22.708376 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:22.708383 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:22.708385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:22.708656 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:22.708841 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:22.708841 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:22.708841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:22.708932 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:22.709146 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:22.709149 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:22.709245 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.709313 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.709417 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260809 19:23:22.709986 31088 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36485
I20260809 19:23:22.710069 31088 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:22.710232 31088 heartbeater.cc:511] Master 127.9.91.126:36485 requested a full tablet report, sending...
I20260809 19:23:22.710568 30425 ts_manager.cc:194] Registered new tserver with Master: 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487)
I20260809 19:23:22.710892 30425 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:59229
I20260809 19:23:22.719269 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:23:22.725656 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.720475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.9.91.126:36485)
0809 19:23:22.720550 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.720552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.720649 (+ 97us) server_negotiation.cc:407] Connection header received
0809 19:23:22.720687 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.720688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.720695 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.720735 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:22.721329 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.721771 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.722406 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.722578 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.722617 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.723655 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:22.723656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:22.723656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:22.723662 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:22.724870 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.725127 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.725128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.725128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.725170 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:22.725278 (+ 108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:22.725279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:22.725279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:22.725390 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:22.725453 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.725535 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.725587 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:23:22.726863 30425 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:23:22.728421 30425 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:23:22.734463 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.730594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.9.91.66:37103)
0809 19:23:22.730755 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.730758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.730766 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:22.730795 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.730797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.730809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.730876 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:22.731223 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.732294 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.733341 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.734010 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.734158 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.734262 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.734347 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.734386 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260809 19:23:22.734503 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.730364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:37103 (local address 127.0.0.1:44554)
0809 19:23:22.730642 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.730654 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.730713 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.730998 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.731001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.731004 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:22.731130 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.731132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.732407 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.732408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.733262 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.733264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.734179 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.734180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.734237 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.734269 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.734331 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.734369 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":3,"thread_start_us":57,"threads_started":1}
I20260809 19:23:22.734542 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.730493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.9.91.67:34487)
0809 19:23:22.730645 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.730649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.730859 (+ 210us) server_negotiation.cc:407] Connection header received
0809 19:23:22.730932 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.730934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.730947 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.731016 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:22.731205 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.732259 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.733503 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.733964 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.734038 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.734093 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.734428 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.734471 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260809 19:23:22.734704 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.730314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:44001 (local address 127.0.0.1:36160)
0809 19:23:22.730454 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.730488 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.730553 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.731004 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.731005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.731007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:22.731079 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.731081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.732042 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.732044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.733290 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.733293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.734193 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.734194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.734294 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.734298 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.734330 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.734374 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":21,"thread_start_us":47,"threads_started":1}
I20260809 19:23:22.734702 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.730441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36160 (local address 127.9.91.65:44001)
0809 19:23:22.730822 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.730827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.730838 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:23:22.730881 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.730883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.730898 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.730999 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:22.731179 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.731930 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.733371 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.734074 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.734448 (+ 374us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.734532 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.734562 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.734628 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":103,"threads_started":1}
I20260809 19:23:22.734377 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.730366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:34487 (local address 127.0.0.1:53548)
0809 19:23:22.730789 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.730799 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.730852 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.731078 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.731079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.731081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:22.731127 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.731129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.732366 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.732367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.733420 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.733422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.734054 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.734055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.734135 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.734149 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.734220 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.734301 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":35,"threads_started":1}
I20260809 19:23:22.735440 30608 tablet_service.cc:1467] Processing CreateTablet for tablet 9838fd8612d04e5eafaded99829ef7a0 (DEFAULT_TABLE table=test-table [id=bfabba31713e407b9a9c2f646979ad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:22.735541 30999 tablet_service.cc:1467] Processing CreateTablet for tablet 9838fd8612d04e5eafaded99829ef7a0 (DEFAULT_TABLE table=test-table [id=bfabba31713e407b9a9c2f646979ad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:22.735802 30608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9838fd8612d04e5eafaded99829ef7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:22.735860 30999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9838fd8612d04e5eafaded99829ef7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:22.738417 31110 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Bootstrap starting.
I20260809 19:23:22.738468 31111 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap starting.
I20260809 19:23:22.738668 30797 tablet_service.cc:1467] Processing CreateTablet for tablet 9838fd8612d04e5eafaded99829ef7a0 (DEFAULT_TABLE table=test-table [id=bfabba31713e407b9a9c2f646979ad86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:22.739027 30797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9838fd8612d04e5eafaded99829ef7a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:22.739108 31110 tablet_bootstrap.cc:654] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:22.739120 31111 tablet_bootstrap.cc:654] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:22.739578 31111 log.cc:824] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:22.739653 31110 log.cc:824] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:22.740367 31111 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: No bootstrap required, opened a new log
I20260809 19:23:22.740443 31111 ts_tablet_manager.cc:1397] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:23:22.742164 31118 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Bootstrap starting.
I20260809 19:23:22.742377 31111 raft_consensus.cc:348] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.742506 31111 raft_consensus.cc:374] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:22.742538 31111 raft_consensus.cc:729] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb8ebfd206c4242a3a52b9cd1d63846, State: Initialized, Role: FOLLOWER
I20260809 19:23:22.742656 31111 consensus_queue.cc:260] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.742739 31118 tablet_bootstrap.cc:654] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:22.742947 31111 ts_tablet_manager.cc:1428] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:23:22.742967 30700 heartbeater.cc:503] Master 127.9.91.126:36485 was elected leader, sending a full tablet report...
I20260809 19:23:22.743304 31110 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: No bootstrap required, opened a new log
I20260809 19:23:22.743373 31110 ts_tablet_manager.cc:1397] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 19:23:22.743353 31118 log.cc:824] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:22.744131 31118 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: No bootstrap required, opened a new log
I20260809 19:23:22.744204 31118 ts_tablet_manager.cc:1397] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:23:22.744935 31110 raft_consensus.cc:348] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.745055 31110 raft_consensus.cc:374] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:22.745087 31110 raft_consensus.cc:729] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c51dde9b054c44a26708d22ac22655, State: Initialized, Role: FOLLOWER
I20260809 19:23:22.745198 31110 consensus_queue.cc:260] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.745433 31110 ts_tablet_manager.cc:1428] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:23:22.745483 31088 heartbeater.cc:503] Master 127.9.91.126:36485 was elected leader, sending a full tablet report...
I20260809 19:23:22.745764 31118 raft_consensus.cc:348] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.745962 31118 raft_consensus.cc:374] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:22.746016 31118 raft_consensus.cc:729] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8372f39adb34f1dbb7e6dbac74f836e, State: Initialized, Role: FOLLOWER
I20260809 19:23:22.746132 31118 consensus_queue.cc:260] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.746409 31118 ts_tablet_manager.cc:1428] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:23:22.746425 30896 heartbeater.cc:503] Master 127.9.91.126:36485 was elected leader, sending a full tablet report...
I20260809 19:23:22.783439 31121 raft_consensus.cc:482] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:22.783524 31121 raft_consensus.cc:504] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.783815 31121 leader_election.cc:288] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65:44001), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
I20260809 19:23:22.787285 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.784034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:47931 (local address 127.9.91.66:37103)
0809 19:23:22.784092 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.784093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.784207 (+ 114us) server_negotiation.cc:407] Connection header received
0809 19:23:22.784233 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.784234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.784243 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.784281 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:22.784402 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.785056 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.786361 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.786962 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.787030 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.787039 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.787186 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.787226 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:23:22.787302 31125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.783946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:37103 (local address 127.9.91.67:47931)
0809 19:23:22.784143 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.784150 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.784198 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.784293 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.784294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.784295 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:22.784348 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.784349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.785136 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.785137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.786285 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.786287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.787043 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.787044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.787119 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.787124 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.787178 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.787222 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":62,"thread_start_us":89,"threads_started":1}
I20260809 19:23:22.787396 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.784051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:47205 (local address 127.9.91.65:44001)
0809 19:23:22.784093 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.784095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.784103 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:22.784125 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.784125 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.784135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.784185 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:22.784333 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.785038 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.786360 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.786976 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.787030 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.787040 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.787302 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.787342 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:23:22.787374 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.783946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:44001 (local address 127.9.91.67:47205)
0809 19:23:22.783992 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.784004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:22.784054 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:22.784205 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:22.784206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:22.784207 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:22.784269 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.784270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.785110 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.785110 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.786284 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:22.786287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.787109 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.787110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:22.787212 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.787216 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:23:22.787273 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 19:23:22.787313 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:23:22.787889 30827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9838fd8612d04e5eafaded99829ef7a0" candidate_uuid: "44c51dde9b054c44a26708d22ac22655" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" is_pre_election: true
I20260809 19:23:22.787907 30636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9838fd8612d04e5eafaded99829ef7a0" candidate_uuid: "44c51dde9b054c44a26708d22ac22655" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" is_pre_election: true
I20260809 19:23:22.787989 30827 raft_consensus.cc:2454] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c51dde9b054c44a26708d22ac22655 in term 0.
I20260809 19:23:22.788023 30636 raft_consensus.cc:2454] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44c51dde9b054c44a26708d22ac22655 in term 0.
I20260809 19:23:22.788199 30925 leader_election.cc:302] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c51dde9b054c44a26708d22ac22655, e8372f39adb34f1dbb7e6dbac74f836e; no voters:
I20260809 19:23:22.788326 31121 raft_consensus.cc:2781] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:23:22.788389 31121 raft_consensus.cc:482] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:23:22.788406 31121 raft_consensus.cc:3037] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:22.789125 31121 raft_consensus.cc:504] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.789232 31121 leader_election.cc:288] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [CANDIDATE]: Term 1 election: Requested vote from peers 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65:44001), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
I20260809 19:23:22.789387 30636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9838fd8612d04e5eafaded99829ef7a0" candidate_uuid: "44c51dde9b054c44a26708d22ac22655" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846"
I20260809 19:23:22.789424 30827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9838fd8612d04e5eafaded99829ef7a0" candidate_uuid: "44c51dde9b054c44a26708d22ac22655" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8372f39adb34f1dbb7e6dbac74f836e"
I20260809 19:23:22.789467 30636 raft_consensus.cc:3037] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:22.789489 30827 raft_consensus.cc:3037] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:22.790104 30636 raft_consensus.cc:2454] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c51dde9b054c44a26708d22ac22655 in term 1.
I20260809 19:23:22.790110 30827 raft_consensus.cc:2454] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44c51dde9b054c44a26708d22ac22655 in term 1.
I20260809 19:23:22.790261 30925 leader_election.cc:302] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44c51dde9b054c44a26708d22ac22655, e8372f39adb34f1dbb7e6dbac74f836e; no voters:
I20260809 19:23:22.790343 31121 raft_consensus.cc:2781] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:22.790469 31121 raft_consensus.cc:686] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 1 LEADER]: Becoming Leader. State: Replica: 44c51dde9b054c44a26708d22ac22655, State: Running, Role: LEADER
I20260809 19:23:22.790555 31121 consensus_queue.cc:237] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:22.791260 30425 catalog_manager.cc:5495] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44c51dde9b054c44a26708d22ac22655 (127.9.91.67). New cstate: current_term: 1 leader_uuid: "44c51dde9b054c44a26708d22ac22655" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } health_report { overall_health: HEALTHY } } }
I20260809 19:23:22.805835 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.803764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.9.91.67:34487)
0809 19:23:22.803805 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:22.803807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:22.803815 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:22.803870 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:22.803871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:22.803887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:22.803941 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:23:22.804052 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.804477 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.805193 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:22.805370 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:22.805422 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:22.805566 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:23:22.805652 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:23:22.805720 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:22.805756 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:23:22.805774 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:23:22.807312 30636 raft_consensus.cc:1261] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Refusing update from remote peer 44c51dde9b054c44a26708d22ac22655: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 19:23:22.807397 30897 tablet.cc:2369] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:23:22.807538 30827 raft_consensus.cc:1261] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Refusing update from remote peer 44c51dde9b054c44a26708d22ac22655: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:22.807782 31121 consensus_queue.cc:1035] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:22.807897 31126 consensus_queue.cc:1035] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:22.810264 31136 mvcc.cc:205] Tried to move back new op lower bound from 7316698737896529920 to 7316698737830707200. Current Snapshot: MvccSnapshot[applied={T|T < 7316698737896529920}]
I20260809 19:23:22.811203 31135 mvcc.cc:205] Tried to move back new op lower bound from 7316698737896529920 to 7316698737830707200. Current Snapshot: MvccSnapshot[applied={T|T < 7316698737896529920}]
I20260809 19:23:22.812244 30423 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:22.812317 30423 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:22.812958 30423 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Sending DeleteTablet for 3 replicas of tablet 9838fd8612d04e5eafaded99829ef7a0
I20260809 19:23:22.813278 30608 tablet_service.cc:1514] Processing DeleteTablet for tablet 9838fd8612d04e5eafaded99829ef7a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:22 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:36160
I20260809 19:23:22.813287 30797 tablet_service.cc:1514] Processing DeleteTablet for tablet 9838fd8612d04e5eafaded99829ef7a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:22 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:44554
I20260809 19:23:22.813494 30999 tablet_service.cc:1514] Processing DeleteTablet for tablet 9838fd8612d04e5eafaded99829ef7a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:22 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53548
I20260809 19:23:22.813599 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 30515
I20260809 19:23:22.813735 31149 tablet_replica.cc:330] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: stopping tablet replica
I20260809 19:23:22.813861 31149 raft_consensus.cc:2229] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:23:22.814021 31149 raft_consensus.cc:2258] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:22.818182 31147 tablet_replica.cc:330] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: stopping tablet replica
I20260809 19:23:22.818351 31147 raft_consensus.cc:2229] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:22.818491 31147 raft_consensus.cc:2258] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:22.818984 31149 ts_tablet_manager.cc:1905] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:22.820783 31149 ts_tablet_manager.cc:1918] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:22.820909 31149 log.cc:1196] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/9838fd8612d04e5eafaded99829ef7a0
I20260809 19:23:22.821259 31149 ts_tablet_manager.cc:1939] T 9838fd8612d04e5eafaded99829ef7a0 P 44c51dde9b054c44a26708d22ac22655: Deleting consensus metadata
I20260809 19:23:22.821607 31147 ts_tablet_manager.cc:1905] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:22.821727 30402 catalog_manager.cc:4841] TS 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487): tablet 9838fd8612d04e5eafaded99829ef7a0 (table test-table [id=bfabba31713e407b9a9c2f646979ad86]) successfully deleted
I20260809 19:23:22.822949 31147 ts_tablet_manager.cc:1918] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:22.823024 31147 log.cc:1196] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/9838fd8612d04e5eafaded99829ef7a0
I20260809 19:23:22.823249 31147 ts_tablet_manager.cc:1939] T 9838fd8612d04e5eafaded99829ef7a0 P e8372f39adb34f1dbb7e6dbac74f836e: Deleting consensus metadata
I20260809 19:23:22.823601 30399 catalog_manager.cc:4841] TS e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103): tablet 9838fd8612d04e5eafaded99829ef7a0 (table test-table [id=bfabba31713e407b9a9c2f646979ad86]) successfully deleted
W20260809 19:23:22.826416 30403 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:44001 (error 108)
I20260809 19:23:22.826614 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 30706
I20260809 19:23:22.828043 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:22.826715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:44001 (local address 127.0.0.1:36168)
0809 19:23:22.827960 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:22.828008 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1220}
W20260809 19:23:22.833137 30403 catalog_manager.cc:4584] TS 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65:44001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9838fd8612d04e5eafaded99829ef7a0: Network error: Client connection negotiation failed: client connection to 127.9.91.65:44001: connect: Connection refused (error 111)
I20260809 19:23:22.850337 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 30903
I20260809 19:23:22.860750 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 30368
I20260809 19:23:22.868909 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36485
--webserver_interface=127.9.91.126
--webserver_port=41305
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:22.947098 31153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:22.947269 31153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:22.947290 31153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:22.948376 31153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:22.948414 31153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:22.948427 31153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:22.948451 31153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:22.948467 31153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:22.949874 31153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=41305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:22.950080 31153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:22.950284 31153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:22.958702 31161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.958760 31162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:22.958758 31164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:22.958787 31153 server_base.cc:1034] running on GCE node
I20260809 19:23:22.959153 31153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:22.959447 31153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:22.960588 31153 hybrid_clock.cc:648] HybridClock initialized: now 1786303402960571 us; error 29 us; skew 500 ppm
Aug 09 19:23:22 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303402, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:22.962982 31153 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:22.963411 31153 webserver.cc:460] Webserver started at http://127.9.91.126:41305/ using document root <none> and password file <none>
I20260809 19:23:22.963617 31153 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:22.963678 31153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:22.964915 31153 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:22.965472 31181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:22.965622 31153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:22.965704 31153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "ed658dde74f6447daf5932837e4453a5"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "c67f24e9b4b226b577092fba17705758"
server_key_iv: "d897bb7d37bc5efbc14243111cd72ce3"
server_key_version: "encryptionkey@0"
I20260809 19:23:22.966035 31153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:22.995903 31153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:22.997484 31153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:22.997617 31153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:23.001545 31153 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:36485
I20260809 19:23:23.001598 31256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:36485 every 8 connection(s)
I20260809 19:23:23.001931 31153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:23.003631 31258 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:23:23.003750 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 31153
I20260809 19:23:23.004081 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:44001
--local_ip_for_outbound_sockets=127.9.91.65
--tserver_master_addrs=127.9.91.126:36485
--webserver_port=34573
--webserver_interface=127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:23.004527 31258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Bootstrap starting.
I20260809 19:23:23.013881 31258 log.cc:824] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:23.015897 31258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:23:23.016184 31258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Bootstrap complete.
I20260809 19:23:23.018364 31258 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.018603 31258 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed658dde74f6447daf5932837e4453a5, State: Initialized, Role: FOLLOWER
I20260809 19:23:23.018745 31258 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.018807 31258 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:23:23.018854 31258 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:23:23.018908 31258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:23:23.020021 31258 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.020117 31258 leader_election.cc:302] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed658dde74f6447daf5932837e4453a5; no voters:
I20260809 19:23:23.020278 31258 leader_election.cc:288] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:23:23.020328 31266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:23:23.020452 31266 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 LEADER]: Becoming Leader. State: Replica: ed658dde74f6447daf5932837e4453a5, State: Running, Role: LEADER
I20260809 19:23:23.020542 31258 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:23.020673 31266 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.020965 31268 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ed658dde74f6447daf5932837e4453a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } } }
I20260809 19:23:23.021049 31268 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:23.021365 31270 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed658dde74f6447daf5932837e4453a5. Latest consensus state: current_term: 2 leader_uuid: "ed658dde74f6447daf5932837e4453a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } } }
I20260809 19:23:23.021428 31270 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:23.021638 31277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:23:23.022418 31277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:23:23.022593 31277 catalog_manager.cc:1259] Loaded cluster ID: b0b9e61a09d5478e98562b1a8444780a
I20260809 19:23:23.022773 31277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:23:23.023104 31277 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 19:23:23.027338 31277 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 19:23:23.029589 31277 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:23:23.029785 31277 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Loaded TSK: 0
I20260809 19:23:23.030005 31277 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:23:23.085510 31262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:23.085701 31262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:23.085733 31262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:23.086809 31262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:23.086848 31262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:23.086882 31262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:23:23.088310 31262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:44001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=34573
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:23.088532 31262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:23.088726 31262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:23.094408 31298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.094444 31297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.094470 31262 server_base.cc:1034] running on GCE node
W20260809 19:23:23.094591 31300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.094926 31262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:23.095157 31262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:23.096295 31262 hybrid_clock.cc:648] HybridClock initialized: now 1786303403096285 us; error 33 us; skew 500 ppm
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:23.098791 31262 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:23:23.099244 31262 webserver.cc:460] Webserver started at http://127.9.91.65:34573/ using document root <none> and password file <none>
I20260809 19:23:23.099450 31262 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:23.099515 31262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:23.100840 31262 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.101502 31316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.101670 31262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.101754 31262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "4cb8ebfd206c4242a3a52b9cd1d63846"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "d383dc58078dd04081d40178675de485"
server_key_iv: "233ce102fbd80623bd4053e1f21c98ae"
server_key_version: "encryptionkey@0"
I20260809 19:23:23.102033 31262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:23.110316 31262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:23.112243 31262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:23.112367 31262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:23.112550 31262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:23.112932 31330 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:23:23.114886 31262 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:23:23.114965 31262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:23:23.114998 31262 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:23:23.115578 31262 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:23:23.115613 31262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:23:23.115688 31330 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap starting.
I20260809 19:23:23.122383 31262 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:44001
I20260809 19:23:23.122722 31262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:23:23.127755 31330 log.cc:824] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:23.128083 31492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:44001 every 8 connection(s)
I20260809 19:23:23.129756 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 31262
I20260809 19:23:23.130031 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:37103
--local_ip_for_outbound_sockets=127.9.91.66
--tserver_master_addrs=127.9.91.126:36485
--webserver_port=37435
--webserver_interface=127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:23.130548 31330 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:23:23.130861 31330 tablet_bootstrap.cc:492] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap complete.
I20260809 19:23:23.132074 31330 ts_tablet_manager.cc:1397] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.012s
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:23.133461 31330 raft_consensus.cc:348] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:23.134683 31330 raft_consensus.cc:729] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb8ebfd206c4242a3a52b9cd1d63846, State: Initialized, Role: FOLLOWER
I20260809 19:23:23.134907 31330 consensus_queue.cc:260] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } }
I20260809 19:23:23.135290 31330 ts_tablet_manager.cc:1428] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:23:23.138907 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.123370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:35771)
0809 19:23:23.123617 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.123660 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.124171 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.128528 (+ 4357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.128532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.128824 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:23.128999 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.129003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.130417 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.130418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.131573 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.131577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.132373 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.132374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.132523 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.133086 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:23.133096 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:23.134862 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:23.137088 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:23.137096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:23.137099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:23.137376 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:23.137663 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:23.137664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:23.137664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:23.137802 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:23.138288 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:23.138296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:23.138447 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.138552 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.138713 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":128,"threads_started":1}
I20260809 19:23:23.139465 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.123476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35771 (local address 127.9.91.126:36485)
0809 19:23:23.128138 (+ 4662us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.128147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.128186 (+ 39us) server_negotiation.cc:407] Connection header received
0809 19:23:23.128313 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.128320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.128370 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.128492 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:23.129115 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.130285 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.131696 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.132269 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.132367 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.135011 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:23.135016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:23.135016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:23.135028 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:23.136982 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.137470 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.137474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.137474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.137585 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.137878 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.137879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.137880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.138179 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:23.138257 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.139127 (+ 870us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.139263 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4582,"thread_start_us":91,"threads_started":1}
I20260809 19:23:23.140146 31494 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36485
I20260809 19:23:23.140244 31494 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:23.140416 31494 heartbeater.cc:511] Master 127.9.91.126:36485 requested a full tablet report, sending...
I20260809 19:23:23.141204 31201 ts_manager.cc:194] Registered new tserver with Master: 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65:44001)
I20260809 19:23:23.142381 31201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:35771
I20260809 19:23:23.143395 31494 heartbeater.cc:503] Master 127.9.91.126:36485 was elected leader, sending a full tablet report...
I20260809 19:23:23.147725 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.143127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.9.91.65:44001)
0809 19:23:23.143533 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.143537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.143548 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:23:23.143581 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.143582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.143597 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.143855 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:23.144061 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.145107 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.146595 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.147328 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.147398 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.147456 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.147628 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.147663 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":76,"threads_started":1}
I20260809 19:23:23.147755 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.142834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:44001 (local address 127.0.0.1:36182)
0809 19:23:23.143289 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.143324 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.143390 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.143919 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.143921 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.143923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:23.143989 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.143990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.145191 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.145192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.146516 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.146519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.147414 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.147415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.147521 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.147536 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.147617 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.147666 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":131,"threads_started":1}
I20260809 19:23:23.148286 31397 tablet_service.cc:1514] Processing DeleteTablet for tablet 9838fd8612d04e5eafaded99829ef7a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:22 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:36182
I20260809 19:23:23.148806 31512 tablet_replica.cc:330] stopping tablet replica
I20260809 19:23:23.148926 31512 raft_consensus.cc:2229] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:23.148993 31512 raft_consensus.cc:2258] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:23.149283 31512 ts_tablet_manager.cc:1905] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:23.151532 31512 ts_tablet_manager.cc:1918] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:23.151616 31512 log.cc:1196] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/9838fd8612d04e5eafaded99829ef7a0
I20260809 19:23:23.151929 31512 ts_tablet_manager.cc:1939] T 9838fd8612d04e5eafaded99829ef7a0 P 4cb8ebfd206c4242a3a52b9cd1d63846: Deleting consensus metadata
I20260809 19:23:23.152428 31186 catalog_manager.cc:4841] TS 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65:44001): tablet 9838fd8612d04e5eafaded99829ef7a0 (table test-table [id=bfabba31713e407b9a9c2f646979ad86]) successfully deleted
W20260809 19:23:23.215751 31503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:23.215932 31503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:23.215962 31503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:23.217046 31503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:23.217087 31503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:23.217121 31503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:23:23.218537 31503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:37103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=37435
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:23.218778 31503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:23.218992 31503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:23.223755 31520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.223788 31517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.223824 31503 server_base.cc:1034] running on GCE node
W20260809 19:23:23.223783 31522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.224074 31503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:23.224306 31503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:23.225455 31503 hybrid_clock.cc:648] HybridClock initialized: now 1786303403225439 us; error 27 us; skew 500 ppm
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:23.227891 31503 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:23:23.228291 31503 webserver.cc:460] Webserver started at http://127.9.91.66:37435/ using document root <none> and password file <none>
I20260809 19:23:23.228489 31503 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:23.228551 31503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:23.229746 31503 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.230270 31532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.230423 31503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.230502 31503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "e8372f39adb34f1dbb7e6dbac74f836e"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "9aaf1045dbd03cb6fc277770d83ee6c4"
server_key_iv: "fcc3a3bd4608729eec53426f2000c3e8"
server_key_version: "encryptionkey@0"
I20260809 19:23:23.230785 31503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:23.239732 31503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:23.241627 31503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:23.241750 31503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:23.241927 31503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:23.242175 31503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:23.242208 31503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.242231 31503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:23.242259 31503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.247830 31503 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:37103
I20260809 19:23:23.247871 31710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:37103 every 8 connection(s)
I20260809 19:23:23.248212 31503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:23.256048 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.248963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:48987 (local address 127.9.91.126:36485)
0809 19:23:23.249021 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.249022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.249139 (+ 117us) server_negotiation.cc:407] Connection header received
0809 19:23:23.249537 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.249538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.249550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.249602 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:23.250402 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.250899 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.251804 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.251994 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.252048 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.253883 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:23.253885 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:23.253885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:23.253892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:23.255000 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.255328 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.255328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.255329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.255376 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.255538 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.255538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.255539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.255635 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:23.255679 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.255908 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.255986 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:23:23.256086 31713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.248891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:48987)
0809 19:23:23.249031 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.249068 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.249457 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.249648 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.249650 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.250152 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:23.250320 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.250324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.250979 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.250980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.251740 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.251743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.252044 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.252045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.252132 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.252533 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:23.252541 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:23.253776 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:23.255076 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:23.255081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:23.255082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:23.255272 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:23.255417 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:23.255417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:23.255418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:23.255498 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:23.255675 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:23.255678 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:23.255773 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.255848 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.255951 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260809 19:23:23.256508 31711 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36485
I20260809 19:23:23.256592 31711 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:23.256742 31711 heartbeater.cc:511] Master 127.9.91.126:36485 requested a full tablet report, sending...
I20260809 19:23:23.257073 31201 ts_manager.cc:194] Registered new tserver with Master: e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
I20260809 19:23:23.257421 31201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:48987
I20260809 19:23:23.257898 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 31503
I20260809 19:23:23.258183 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:34487
--local_ip_for_outbound_sockets=127.9.91.67
--tserver_master_addrs=127.9.91.126:36485
--webserver_port=39901
--webserver_interface=127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:23.340258 31714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:23.340426 31714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:23.340447 31714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:23.341526 31714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:23.341567 31714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:23.341593 31714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:23:23.342990 31714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:34487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=39901
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:23.343199 31714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:23.343392 31714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:23.347870 31728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.347886 31721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.347957 31714 server_base.cc:1034] running on GCE node
W20260809 19:23:23.347872 31722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.348272 31714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:23.348482 31714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:23.349617 31714 hybrid_clock.cc:648] HybridClock initialized: now 1786303403349609 us; error 25 us; skew 500 ppm
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:23.351972 31714 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:23:23.352383 31714 webserver.cc:460] Webserver started at http://127.9.91.67:39901/ using document root <none> and password file <none>
I20260809 19:23:23.352609 31714 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:23.352675 31714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:23.354000 31714 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:23:23.354560 31738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.354724 31714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:23:23.354818 31714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "44c51dde9b054c44a26708d22ac22655"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "f725af70fa71a3d29f254dae78dec019"
server_key_iv: "33097e4f12755f173d8d16cd3d72f44c"
server_key_version: "encryptionkey@0"
I20260809 19:23:23.355154 31714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:23.374965 31714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:23.376596 31714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:23.376736 31714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:23.376933 31714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:23.377211 31714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:23.377249 31714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.377278 31714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:23.377305 31714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.383569 31714 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:34487
I20260809 19:23:23.383605 31916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:34487 every 8 connection(s)
I20260809 19:23:23.383911 31714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:23.391762 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.384547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:39573 (local address 127.9.91.126:36485)
0809 19:23:23.384603 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.384605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.384813 (+ 208us) server_negotiation.cc:407] Connection header received
0809 19:23:23.385219 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.385221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.385232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.385287 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:23.386066 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.386549 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.387480 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.387684 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.387743 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.389550 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:23.389551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:23.389551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:23.389557 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:23.390694 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.391067 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.391067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.391068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.391125 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.391301 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.391302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.391302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.391400 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:23.391448 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.391633 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.391696 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 19:23:23.391861 31920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.384538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:39573)
0809 19:23:23.384695 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.384725 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.385135 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.385333 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.385336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.385831 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:23.385999 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.386003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.386613 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.386614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.387401 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.387404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.387746 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.387747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.387833 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.388251 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:23.388259 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:23.389472 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:23.390768 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:23.390773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:23.390775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:23.391009 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:23.391177 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:23.391177 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:23.391177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:23.391264 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:23.391442 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:23.391444 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:23.391540 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.391618 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.391721 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260809 19:23:23.392321 31917 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36485
I20260809 19:23:23.392393 31917 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:23.392534 31917 heartbeater.cc:511] Master 127.9.91.126:36485 requested a full tablet report, sending...
I20260809 19:23:23.392880 31201 ts_manager.cc:194] Registered new tserver with Master: 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487)
I20260809 19:23:23.393217 31201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:39573
I20260809 19:23:23.393461 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 31714
I20260809 19:23:23.398875 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.393835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.9.91.126:36485)
0809 19:23:23.393874 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.393875 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.393974 (+ 99us) server_negotiation.cc:407] Connection header received
0809 19:23:23.394053 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.394053 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.394060 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.394102 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:23.394603 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.395072 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.395729 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.395893 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.395931 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.396901 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:23.396902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:23.396903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:23.396908 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:23.398033 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.398356 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.398356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.398356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.398399 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:23.398524 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:23.398524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:23.398524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:23.398610 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:23.398722 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.398769 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.398821 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:23:23.399624 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:23:23.401903 31498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.399914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60224 (local address 127.9.91.126:36485)
0809 19:23:23.399971 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.399973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.400041 (+ 68us) server_negotiation.cc:407] Connection header received
0809 19:23:23.400113 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.400114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.400144 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.400195 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:23:23.400274 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.400688 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.401357 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.401520 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.401558 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.401674 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:23:23.401752 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:23:23.401801 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.401837 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.401850 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:23:23.402187 31201 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:23:23.403806 31201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:23:23.405818 31397 tablet_service.cc:1467] Processing CreateTablet for tablet d83cf046896748969f5591a535101c21 (DEFAULT_TABLE table=test-table [id=b04d565cdc6a4938ab482fcc61a07b55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:23.406002 31397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83cf046896748969f5591a535101c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:23.407315 31330 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap starting.
I20260809 19:23:23.407657 31330 tablet_bootstrap.cc:654] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:23.408277 31330 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: No bootstrap required, opened a new log
I20260809 19:23:23.408339 31330 ts_tablet_manager.cc:1397] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.408459 31330 raft_consensus.cc:348] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.408545 31330 raft_consensus.cc:374] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:23.408587 31330 raft_consensus.cc:729] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb8ebfd206c4242a3a52b9cd1d63846, State: Initialized, Role: FOLLOWER
I20260809 19:23:23.408632 31330 consensus_queue.cc:260] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.408725 31330 ts_tablet_manager.cc:1428] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.408908 31509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.405436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:34487 (local address 127.0.0.1:53572)
0809 19:23:23.405475 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.405484 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.405528 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.405828 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.405829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.405831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:23.405869 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.405870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.407017 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.407018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.407878 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.407882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.408693 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.408694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.408757 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.408760 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.408804 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.408845 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 19:23:23.409118 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.405496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.9.91.67:34487)
0809 19:23:23.405655 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.405659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.405669 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:23:23.405703 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.405705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.405722 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.405831 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:23.405951 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.406896 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.407977 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.408553 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.408608 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.408664 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.409003 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.409040 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260809 19:23:23.409618 31815 tablet_service.cc:1467] Processing CreateTablet for tablet d83cf046896748969f5591a535101c21 (DEFAULT_TABLE table=test-table [id=b04d565cdc6a4938ab482fcc61a07b55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:23.409844 31815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83cf046896748969f5591a535101c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:23.409904 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.405462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:37103 (local address 127.0.0.1:44562)
0809 19:23:23.405694 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.405704 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.405755 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.406156 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.406157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.406160 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:23.406202 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.406204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.407784 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.407785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.408841 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.408845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.409663 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.409664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.409728 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.409732 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.409792 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.409832 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":51,"threads_started":1}
I20260809 19:23:23.409925 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.405657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.9.91.66:37103)
0809 19:23:23.405971 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.405976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.405987 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:23:23.406051 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.406053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.406069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.406158 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:23.406266 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.407644 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.408950 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.409578 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.409651 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.409705 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.409807 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.409845 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":107,"threads_started":1}
I20260809 19:23:23.410504 31601 tablet_service.cc:1467] Processing CreateTablet for tablet d83cf046896748969f5591a535101c21 (DEFAULT_TABLE table=test-table [id=b04d565cdc6a4938ab482fcc61a07b55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:23.410751 31601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83cf046896748969f5591a535101c21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:23.412420 31928 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Bootstrap starting.
I20260809 19:23:23.413110 31928 tablet_bootstrap.cc:654] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:23.413822 31929 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Bootstrap starting.
I20260809 19:23:23.413726 31928 log.cc:824] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:23.414414 31929 tablet_bootstrap.cc:654] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:23.414703 31928 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: No bootstrap required, opened a new log
I20260809 19:23:23.414765 31928 ts_tablet_manager.cc:1397] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:23:23.414897 31929 log.cc:824] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:23.415611 31929 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: No bootstrap required, opened a new log
I20260809 19:23:23.415669 31929 ts_tablet_manager.cc:1397] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:23:23.416378 31928 raft_consensus.cc:348] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.416517 31928 raft_consensus.cc:374] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:23.416543 31928 raft_consensus.cc:729] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c51dde9b054c44a26708d22ac22655, State: Initialized, Role: FOLLOWER
I20260809 19:23:23.416646 31928 consensus_queue.cc:260] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.416903 31928 ts_tablet_manager.cc:1428] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:23:23.416942 31917 heartbeater.cc:503] Master 127.9.91.126:36485 was elected leader, sending a full tablet report...
I20260809 19:23:23.417120 31929 raft_consensus.cc:348] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.417285 31929 raft_consensus.cc:374] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:23.417330 31929 raft_consensus.cc:729] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8372f39adb34f1dbb7e6dbac74f836e, State: Initialized, Role: FOLLOWER
I20260809 19:23:23.417443 31929 consensus_queue.cc:260] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.417725 31929 ts_tablet_manager.cc:1428] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:23:23.417753 31711 heartbeater.cc:503] Master 127.9.91.126:36485 was elected leader, sending a full tablet report...
I20260809 19:23:23.421630 31504 raft_consensus.cc:482] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:23.421753 31504 raft_consensus.cc:504] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.422005 31504 leader_election.cc:288] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
I20260809 19:23:23.425391 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.422285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:46747 (local address 127.9.91.66:37103)
0809 19:23:23.422335 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.422337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.422345 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:23.422430 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.422431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.422438 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.422472 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:23.422608 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.423288 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.424516 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.425057 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.425131 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.425140 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.425303 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.425334 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:23:23.425391 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.422205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:56233 (local address 127.9.91.67:34487)
0809 19:23:23.422236 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.422238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.422243 (+ 5us) server_negotiation.cc:407] Connection header received
0809 19:23:23.422261 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.422262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.422270 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.422317 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:23.422516 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.423200 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.424511 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.425057 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.425127 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.425134 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.425303 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.425334 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:23:23.425447 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.422105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:34487 (local address 127.9.91.65:56233)
0809 19:23:23.422141 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.422151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.422200 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.422378 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.422379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.422381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:23.422444 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.422446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.423268 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.423269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.424424 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.424427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.425135 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.425136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.425227 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.425235 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.425298 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.425347 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":16}
I20260809 19:23:23.425460 31944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.422153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:37103 (local address 127.9.91.65:46747)
0809 19:23:23.422293 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.422304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.422360 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.422478 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:23.422479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:23.422481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:23.422548 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.422550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.423348 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.423349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.424436 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:23.424438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.425136 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.425137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:23.425227 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.425236 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 19:23:23.425298 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:23:23.425346 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":6,"thread_start_us":85,"threads_started":1}
I20260809 19:23:23.425752 31636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83cf046896748969f5591a535101c21" candidate_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" is_pre_election: true
I20260809 19:23:23.425860 31636 raft_consensus.cc:2454] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb8ebfd206c4242a3a52b9cd1d63846 in term 0.
I20260809 19:23:23.425911 31849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83cf046896748969f5591a535101c21" candidate_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c51dde9b054c44a26708d22ac22655" is_pre_election: true
I20260809 19:23:23.425999 31849 raft_consensus.cc:2454] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb8ebfd206c4242a3a52b9cd1d63846 in term 0.
I20260809 19:23:23.426065 31318 leader_election.cc:302] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cb8ebfd206c4242a3a52b9cd1d63846, e8372f39adb34f1dbb7e6dbac74f836e; no voters:
I20260809 19:23:23.426184 31504 raft_consensus.cc:2781] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:23:23.426249 31504 raft_consensus.cc:482] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:23:23.426271 31504 raft_consensus.cc:3037] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:23.426955 31504 raft_consensus.cc:504] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.427083 31504 leader_election.cc:288] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 1 election: Requested vote from peers 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
I20260809 19:23:23.427240 31636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83cf046896748969f5591a535101c21" candidate_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8372f39adb34f1dbb7e6dbac74f836e"
I20260809 19:23:23.427234 31849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83cf046896748969f5591a535101c21" candidate_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44c51dde9b054c44a26708d22ac22655"
I20260809 19:23:23.427301 31849 raft_consensus.cc:3037] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:23.427301 31636 raft_consensus.cc:3037] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:23.428014 31849 raft_consensus.cc:2454] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb8ebfd206c4242a3a52b9cd1d63846 in term 1.
I20260809 19:23:23.428014 31636 raft_consensus.cc:2454] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb8ebfd206c4242a3a52b9cd1d63846 in term 1.
I20260809 19:23:23.428169 31318 leader_election.cc:302] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cb8ebfd206c4242a3a52b9cd1d63846, e8372f39adb34f1dbb7e6dbac74f836e; no voters:
I20260809 19:23:23.428269 31504 raft_consensus.cc:2781] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:23.428398 31504 raft_consensus.cc:686] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 LEADER]: Becoming Leader. State: Replica: 4cb8ebfd206c4242a3a52b9cd1d63846, State: Running, Role: LEADER
I20260809 19:23:23.428464 31504 consensus_queue.cc:237] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.429071 31200 catalog_manager.cc:5495] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65). New cstate: current_term: 1 leader_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } health_report { overall_health: HEALTHY } } }
I20260809 19:23:23.436630 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.434828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.9.91.65:44001)
0809 19:23:23.434872 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:23.434873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:23.434880 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:23:23.434939 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:23.434940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:23.434956 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:23.435008 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:23:23.435101 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.435510 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.436124 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:23.436287 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:23.436327 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:23.436424 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:23:23.436499 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:23:23.436538 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:23.436572 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 19:23:23.436586 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:23:23.437784 31849 raft_consensus.cc:1261] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 1 FOLLOWER]: Refusing update from remote peer 4cb8ebfd206c4242a3a52b9cd1d63846: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:23.437806 31636 raft_consensus.cc:1261] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Refusing update from remote peer 4cb8ebfd206c4242a3a52b9cd1d63846: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:23.438082 31504 consensus_queue.cc:1035] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:23.438201 31945 consensus_queue.cc:1035] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:23.441212 31951 mvcc.cc:205] Tried to move back new op lower bound from 7316698740479381504 to 7316698740443496448. Current Snapshot: MvccSnapshot[applied={T|T < 7316698740479381504}]
I20260809 19:23:23.442266 31201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:23.442337 31201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:23.442899 31201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Sending DeleteTablet for 3 replicas of tablet d83cf046896748969f5591a535101c21
I20260809 19:23:23.443239 31815 tablet_service.cc:1514] Processing DeleteTablet for tablet d83cf046896748969f5591a535101c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:23 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53572
I20260809 19:23:23.443246 31601 tablet_service.cc:1514] Processing DeleteTablet for tablet d83cf046896748969f5591a535101c21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:23 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:44562
I20260809 19:23:23.443305 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 31262
I20260809 19:23:23.443462 31964 tablet_replica.cc:330] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: stopping tablet replica
I20260809 19:23:23.443547 31964 raft_consensus.cc:2229] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:23.443643 31964 raft_consensus.cc:2258] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:23.445792 31963 tablet_replica.cc:330] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: stopping tablet replica
I20260809 19:23:23.445909 31963 raft_consensus.cc:2229] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:23.446017 31963 raft_consensus.cc:2258] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:23.447201 31964 ts_tablet_manager.cc:1905] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:23.449015 31963 ts_tablet_manager.cc:1905] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:23.449141 31964 ts_tablet_manager.cc:1918] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:23.449229 31964 log.cc:1196] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/d83cf046896748969f5591a535101c21
I20260809 19:23:23.449674 31964 ts_tablet_manager.cc:1939] T d83cf046896748969f5591a535101c21 P e8372f39adb34f1dbb7e6dbac74f836e: Deleting consensus metadata
I20260809 19:23:23.450191 31182 catalog_manager.cc:4841] TS e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103): tablet d83cf046896748969f5591a535101c21 (table test-table [id=b04d565cdc6a4938ab482fcc61a07b55]) successfully deleted
I20260809 19:23:23.450372 31963 ts_tablet_manager.cc:1918] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:23.450421 31963 log.cc:1196] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/d83cf046896748969f5591a535101c21
I20260809 19:23:23.450639 31963 ts_tablet_manager.cc:1939] T d83cf046896748969f5591a535101c21 P 44c51dde9b054c44a26708d22ac22655: Deleting consensus metadata
I20260809 19:23:23.450982 31184 catalog_manager.cc:4841] TS 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487): tablet d83cf046896748969f5591a535101c21 (table test-table [id=b04d565cdc6a4938ab482fcc61a07b55]) successfully deleted
W20260809 19:23:23.463937 31186 connection.cc:535] client connection to 127.9.91.65:44001 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 19:23:23.464038 31186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:23:23.464397 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 31503
I20260809 19:23:23.464507 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.464398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:44001 (local address 127.0.0.1:36206)
0809 19:23:23.464442 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.464478 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:44001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 19:23:23.467832 31186 catalog_manager.cc:4584] TS 4cb8ebfd206c4242a3a52b9cd1d63846 (127.9.91.65:44001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d83cf046896748969f5591a535101c21: Network error: Client connection negotiation failed: client connection to 127.9.91.65:44001: connect: Connection refused (error 111)
I20260809 19:23:23.474967 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 31714
I20260809 19:23:23.488202 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 31153
I20260809 19:23:23.495301 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36485
--webserver_interface=127.9.91.126
--webserver_port=41305
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:23.574442 31970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:23.574621 31970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:23.574641 31970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:23.575717 31970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:23.575750 31970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:23.575762 31970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:23.575773 31970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:23.575788 31970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:23.577169 31970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=41305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:23.577345 31970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:23.577504 31970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:23.581832 31982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.581821 31980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.581844 31978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.582206 31970 server_base.cc:1034] running on GCE node
I20260809 19:23:23.582420 31970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:23.582681 31970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:23.583812 31970 hybrid_clock.cc:648] HybridClock initialized: now 1786303403583800 us; error 29 us; skew 500 ppm
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:23.586050 31970 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:23.586452 31970 webserver.cc:460] Webserver started at http://127.9.91.126:41305/ using document root <none> and password file <none>
I20260809 19:23:23.586645 31970 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:23.586690 31970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:23.587908 31970 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.588479 31997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.588626 31970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:23.588717 31970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "ed658dde74f6447daf5932837e4453a5"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "c67f24e9b4b226b577092fba17705758"
server_key_iv: "d897bb7d37bc5efbc14243111cd72ce3"
server_key_version: "encryptionkey@0"
I20260809 19:23:23.588982 31970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:23.605423 31970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:23.607046 31970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:23.607172 31970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:23.610874 31970 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:36485
I20260809 19:23:23.610908 32077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:36485 every 8 connection(s)
I20260809 19:23:23.611249 31970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:23.612658 32080 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:23:23.613207 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Bootstrap starting.
I20260809 19:23:23.619793 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 31970
I20260809 19:23:23.620147 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:44001
--local_ip_for_outbound_sockets=127.9.91.65
--tserver_master_addrs=127.9.91.126:36485
--webserver_port=34573
--webserver_interface=127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:23.622773 32080 log.cc:824] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:23.625392 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:23:23.625706 32080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5: Bootstrap complete.
I20260809 19:23:23.628835 32080 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.629136 32080 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed658dde74f6447daf5932837e4453a5, State: Initialized, Role: FOLLOWER
I20260809 19:23:23.629307 32080 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.629380 32080 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:23:23.629431 32080 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:23:23.629477 32080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:23:23.630771 32080 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.630883 32080 leader_election.cc:302] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed658dde74f6447daf5932837e4453a5; no voters:
I20260809 19:23:23.631068 32080 leader_election.cc:288] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 19:23:23.631121 32086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:23:23.631253 32086 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [term 3 LEADER]: Becoming Leader. State: Replica: ed658dde74f6447daf5932837e4453a5, State: Running, Role: LEADER
I20260809 19:23:23.631321 32080 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:23.631352 32086 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } }
I20260809 19:23:23.631615 32091 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ed658dde74f6447daf5932837e4453a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } } }
I20260809 19:23:23.631703 32091 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:23.631618 32086 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed658dde74f6447daf5932837e4453a5. Latest consensus state: current_term: 3 leader_uuid: "ed658dde74f6447daf5932837e4453a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed658dde74f6447daf5932837e4453a5" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36485 } } }
I20260809 19:23:23.632043 32086 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed658dde74f6447daf5932837e4453a5 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:23.632153 32098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:23:23.632892 32098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:23:23.633002 32098 catalog_manager.cc:1259] Loaded cluster ID: b0b9e61a09d5478e98562b1a8444780a
I20260809 19:23:23.633036 32098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 19:23:23.633560 32098 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f274602a9ad google::LogMessage::Fail() at ??:0
@ 0x7f274602d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f274602a4de google::LogMessage::Flush() at ??:0
@ 0x7f274602ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f274df4b0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f274df548b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2747ef80d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2747eed954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2748d516db start_thread at ??:0
@ 0x7f2744b2861f clone at ??:0
W20260809 19:23:23.714862 32083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:23.715081 32083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:23.715104 32083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:23.716205 32083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:23.716241 32083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:23.716266 32083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:23:23.717644 32083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:44001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=34573
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:23.717919 32083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:23.718144 32083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:23.722807 32140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.722851 32141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.722851 32144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.723152 32083 server_base.cc:1034] running on GCE node
I20260809 19:23:23.723320 32083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:23.723520 32083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:23.724651 32083 hybrid_clock.cc:648] HybridClock initialized: now 1786303403724639 us; error 32 us; skew 500 ppm
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:23.727166 32083 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:23:23.727564 32083 webserver.cc:460] Webserver started at http://127.9.91.65:34573/ using document root <none> and password file <none>
I20260809 19:23:23.727773 32083 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:23.727823 32083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:23.729066 32083 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.729745 32155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.729899 32083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.729969 32083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "4cb8ebfd206c4242a3a52b9cd1d63846"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "d383dc58078dd04081d40178675de485"
server_key_iv: "233ce102fbd80623bd4053e1f21c98ae"
server_key_version: "encryptionkey@0"
I20260809 19:23:23.730259 32083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:23.757990 32083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:23.760005 32083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:23.760142 32083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:23.760346 32083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:23.760761 32169 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:23:23.762665 32083 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:23:23.762712 32083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:23:23.762743 32083 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:23:23.763391 32083 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:23:23.763423 32083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:23:23.763509 32169 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap starting.
I20260809 19:23:23.770153 32083 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:44001
I20260809 19:23:23.770241 32331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:44001 every 8 connection(s)
I20260809 19:23:23.770520 32083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:23:23.775229 32169 log.cc:824] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:23.777205 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 32083
I20260809 19:23:23.777456 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:37103
--local_ip_for_outbound_sockets=127.9.91.66
--tserver_master_addrs=127.9.91.126:36485
--webserver_port=37435
--webserver_interface=127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:23.777784 32169 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:23:23.778079 32169 tablet_bootstrap.cc:492] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Bootstrap complete.
I20260809 19:23:23.779307 32169 ts_tablet_manager.cc:1397] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.012s
I20260809 19:23:23.780764 32169 raft_consensus.cc:348] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.781108 32169 raft_consensus.cc:729] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb8ebfd206c4242a3a52b9cd1d63846, State: Initialized, Role: FOLLOWER
I20260809 19:23:23.781281 32169 consensus_queue.cc:260] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:23.781569 32169 ts_tablet_manager.cc:1428] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260809 19:23:23.859112 32339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:23.859303 32339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:23.859337 32339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:23.860461 32339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:23.860507 32339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:23.860544 32339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:23:23.861951 32339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:37103
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=37435
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:23.862200 32339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:23.862417 32339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:23.867144 32349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.867272 32353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.867190 32339 server_base.cc:1034] running on GCE node
W20260809 19:23:23.867178 32350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.867610 32339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:23.867890 32339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:23.869086 32339 hybrid_clock.cc:648] HybridClock initialized: now 1786303403869072 us; error 29 us; skew 500 ppm
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:23.872121 32339 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:23:23.872649 32339 webserver.cc:460] Webserver started at http://127.9.91.66:37435/ using document root <none> and password file <none>
I20260809 19:23:23.872886 32339 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:23.872946 32339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:23.874585 32339 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:23:23.875281 32362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.875454 32339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:23:23.875562 32339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "e8372f39adb34f1dbb7e6dbac74f836e"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "9aaf1045dbd03cb6fc277770d83ee6c4"
server_key_iv: "fcc3a3bd4608729eec53426f2000c3e8"
server_key_version: "encryptionkey@0"
I20260809 19:23:23.875957 32339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:23.889719 32339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:23.891665 32339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:23.891781 32339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:23.892011 32339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:23.892336 32339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:23.892377 32339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.892410 32339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:23.892433 32339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.900714 32339 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:37103
W20260809 19:23:23.900970 32334 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:23:23.771165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:48349)
0809 19:23:23.771347 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.771379 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:23.771785 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:23.900772 (+128987us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260809 19:23:23.900995 32532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:37103 every 8 connection(s)
W20260809 19:23:23.901197 32332 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:36485 (0 consecutive failures): Network error: Failed to ping master at 127.9.91.126:36485: Client connection negotiation failed: client connection to 127.9.91.126:36485: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:23:23.901249 32339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:23:23.901648 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.901480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:54117)
0809 19:23:23.901566 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.901615 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 19:23:23.901997 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.901895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:46223)
0809 19:23:23.901942 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.901972 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:23:23.902127 32332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:23:23.902295 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.901873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:32773)
0809 19:23:23.902052 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.902136 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
W20260809 19:23:23.902635 32535 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:36485 (0 consecutive failures): Network error: Failed to ping master at 127.9.91.126:36485: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
I20260809 19:23:23.902822 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 32339
I20260809 19:23:23.902904 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.902806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:59519)
0809 19:23:23.902850 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.902881 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:23:23.903123 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:34487
--local_ip_for_outbound_sockets=127.9.91.67
--tserver_master_addrs=127.9.91.126:36485
--webserver_port=39901
--webserver_interface=127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:23:23.903189 32538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:23.903107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:50673)
0809 19:23:23.903144 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:23.903171 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 19:23:23.903326 32535 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:23:23.983810 32539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:23.983990 32539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:23.984012 32539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:23.985080 32539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:23.985111 32539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:23.985134 32539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:23:23.986511 32539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:34487
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=39901
--tserver_master_addrs=127.9.91.126:36485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:23.986693 32539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:23.986867 32539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:23.991513 32549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.991526 32550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:23.991673 32552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:23.991905 32539 server_base.cc:1034] running on GCE node
I20260809 19:23:23.992130 32539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:23.992368 32539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:23.993518 32539 hybrid_clock.cc:648] HybridClock initialized: now 1786303403993500 us; error 29 us; skew 500 ppm
Aug 09 19:23:23 dist-test-slave-1vmg krb5kdc[30347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:23.996112 32539 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:23:23.996510 32539 webserver.cc:460] Webserver started at http://127.9.91.67:39901/ using document root <none> and password file <none>
I20260809 19:23:23.996703 32539 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:23.996747 32539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:23.997953 32539 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:23:23.998507 32562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:23.998669 32539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:23.998751 32539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "44c51dde9b054c44a26708d22ac22655"
format_stamp: "Formatted at 2026-08-09 19:23:22 on dist-test-slave-1vmg"
server_key: "f725af70fa71a3d29f254dae78dec019"
server_key_iv: "33097e4f12755f173d8d16cd3d72f44c"
server_key_version: "encryptionkey@0"
I20260809 19:23:23.999053 32539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:24.014429 32539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:24.016140 32539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:24.016253 32539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:24.016420 32539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:24.016692 32539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:24.016727 32539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:24.016747 32539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:24.016769 32539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:24.022570 32539 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:34487
I20260809 19:23:24.022640 32746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:34487 every 8 connection(s)
I20260809 19:23:24.022944 32539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:23:24.023882 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:24.023535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:49887)
0809 19:23:24.023683 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:24.023747 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
W20260809 19:23:24.024272 32747 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:36485 (0 consecutive failures): Network error: Failed to ping master at 127.9.91.126:36485: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
I20260809 19:23:24.024513 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:24.024427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:52721)
0809 19:23:24.024468 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:24.024493 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:23:24.024761 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:24.024700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:55469)
0809 19:23:24.024731 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:24.024749 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 19:23:24.024840 32747 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:23:24.028254 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 32539
I20260809 19:23:24.902617 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:24.902415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:56597)
0809 19:23:24.902525 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:24.902558 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260809 19:23:24.903757 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:24.903592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:50839)
0809 19:23:24.903686 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:24.903713 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":48,"threads_started":1}
I20260809 19:23:25.025303 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:25.025111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:43211)
0809 19:23:25.025217 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:25.025247 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260809 19:23:25.098367 32757 raft_consensus.cc:482] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:25.098490 32757 raft_consensus.cc:504] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:25.098791 32757 leader_election.cc:288] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
Aug 09 19:23:25 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.67@KRBTEST.COM
Aug 09 19:23:25 dist-test-slave-1vmg krb5kdc[30347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303403, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.66@KRBTEST.COM
I20260809 19:23:25.108965 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:25.099066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:51433 (local address 127.9.91.66:37103)
0809 19:23:25.099207 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:25.099221 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:25.099257 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:23:25.099864 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:25.099867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:25.099934 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:25.100088 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:25.100777 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.101694 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.102820 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.103242 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.103328 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:25.105829 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:25.105832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:25.105833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:25.105842 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:25.107446 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:25.107853 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:25.107856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:25.107856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:25.107940 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:25.108181 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:25.108182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:25.108183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:25.108399 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:25.108470 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:25.108744 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 19:23:25.108855 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260809 19:23:25.108971 32761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:25.099110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:45515 (local address 127.9.91.67:34487)
0809 19:23:25.099302 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:25.099309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:25.099334 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:23:25.099864 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:25.099867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:25.099929 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:25.100095 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:25.100784 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.101694 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.102817 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.103257 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.103374 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:25.105455 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:25.105459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:25.105460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:25.105472 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:25.107383 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:25.107859 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:25.107862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:25.107863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:25.107953 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:25.108180 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:25.108181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:25.108182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:25.108399 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:25.108470 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:25.108744 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 19:23:25.108861 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":87,"threads_started":1}
I20260809 19:23:25.108960 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:25.098944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:37103 (local address 127.9.91.65:51433)
0809 19:23:25.098998 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:25.099009 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:25.099083 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:25.100130 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:25.100133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:25.100464 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:25.100663 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:25.100669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.101784 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.101785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:25.102744 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:25.102747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.103324 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.103325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:25.103434 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:25.104004 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:25.104009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:25.105751 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:25.107522 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:25.107524 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:25.107524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:25.107779 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:25.108001 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:25.108002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:25.108002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:25.108121 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:25.108464 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:25.108471 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:25.108593 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 19:23:25.108680 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 19:23:25.108858 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":42}
I20260809 19:23:25.109053 32760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:25.098944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:34487 (local address 127.9.91.65:45515)
0809 19:23:25.099199 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:25.099208 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:25.099258 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:25.100216 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:25.100217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:25.100465 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:25.100663 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:25.100669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.101803 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.101804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:25.102743 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:25.102746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:25.103348 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:25.103349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:25.103432 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:25.103952 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:25.103960 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:25.105353 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:25.107464 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:25.107472 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:25.107474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:25.107780 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:25.108016 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:25.108017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:25.108017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:25.108122 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:25.108467 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:25.108473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:25.108593 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 19:23:25.108681 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 19:23:25.108932 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":285,"thread_start_us":67,"threads_started":1}
W20260809 19:23:25.109897 32160 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487): Not found: Tablet not found: d83cf046896748969f5591a535101c21
W20260809 19:23:25.110308 32156 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103): Not found: Tablet not found: d83cf046896748969f5591a535101c21
I20260809 19:23:25.110361 32156 leader_election.cc:302] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb8ebfd206c4242a3a52b9cd1d63846; no voters: 44c51dde9b054c44a26708d22ac22655, e8372f39adb34f1dbb7e6dbac74f836e
I20260809 19:23:25.110476 32757 raft_consensus.cc:2726] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:23:25.903239 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:25.903042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:51155)
0809 19:23:25.903160 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:25.903195 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260809 19:23:25.904273 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:25.904094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:40741)
0809 19:23:25.904212 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:25.904238 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260809 19:23:26.025861 32767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:26.025655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:60725)
0809 19:23:26.025787 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:26.025822 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260809 19:23:26.903810 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:26.903619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:60599)
0809 19:23:26.903739 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:26.903771 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260809 19:23:26.904783 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:26.904611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:60131)
0809 19:23:26.904716 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:26.904744 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
I20260809 19:23:27.002835 303 raft_consensus.cc:482] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:27.002954 303 raft_consensus.cc:504] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:27.003132 303 leader_election.cc:288] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
W20260809 19:23:27.003536 32160 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487): Not found: Tablet not found: d83cf046896748969f5591a535101c21
W20260809 19:23:27.003631 32156 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103): Not found: Tablet not found: d83cf046896748969f5591a535101c21
I20260809 19:23:27.003679 32156 leader_election.cc:302] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb8ebfd206c4242a3a52b9cd1d63846; no voters: 44c51dde9b054c44a26708d22ac22655, e8372f39adb34f1dbb7e6dbac74f836e
I20260809 19:23:27.003764 303 raft_consensus.cc:2726] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:23:27.026446 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:27.026238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:42583)
0809 19:23:27.026377 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:27.026409 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260809 19:23:27.904364 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:27.904165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:59041)
0809 19:23:27.904290 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:27.904325 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260809 19:23:27.905282 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:27.905128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:44185)
0809 19:23:27.905221 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:27.905251 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":51,"threads_started":1}
I20260809 19:23:28.027038 309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:28.026814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:48845)
0809 19:23:28.026962 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:28.026995 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":76,"threads_started":1}
I20260809 19:23:28.905010 310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:28.904813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:46293)
0809 19:23:28.904933 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:28.904970 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260809 19:23:28.905764 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:28.905603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:52251)
0809 19:23:28.905701 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:28.905730 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
I20260809 19:23:29.027618 314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:29.027395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:59387)
0809 19:23:29.027541 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:29.027577 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260809 19:23:29.367044 315 raft_consensus.cc:482] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:29.367136 315 raft_consensus.cc:504] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:29.367304 315 leader_election.cc:288] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
W20260809 19:23:29.367655 32160 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487): Not found: Tablet not found: d83cf046896748969f5591a535101c21
W20260809 19:23:29.367755 32156 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103): Not found: Tablet not found: d83cf046896748969f5591a535101c21
I20260809 19:23:29.367795 32156 leader_election.cc:302] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb8ebfd206c4242a3a52b9cd1d63846; no voters: 44c51dde9b054c44a26708d22ac22655, e8372f39adb34f1dbb7e6dbac74f836e
I20260809 19:23:29.367858 315 raft_consensus.cc:2726] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:23:29.905611 317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:29.905400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:60305)
0809 19:23:29.905524 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:29.905564 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260809 19:23:29.906292 320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:29.906120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:44247)
0809 19:23:29.906228 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:29.906257 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260809 19:23:30.028156 321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:30.027976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:51501)
0809 19:23:30.028082 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:30.028112 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260809 19:23:30.906167 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:30.905978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:55221)
0809 19:23:30.906093 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:30.906126 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260809 19:23:30.906811 325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:30.906641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:37165)
0809 19:23:30.906742 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:30.906775 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":51,"threads_started":1}
I20260809 19:23:31.028734 332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:31.028534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:32933)
0809 19:23:31.028663 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:31.028694 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260809 19:23:31.906733 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:31.906519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:37407)
0809 19:23:31.906655 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:31.906690 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260809 19:23:31.907370 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:31.907189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:53809)
0809 19:23:31.907296 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:31.907333 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
I20260809 19:23:32.029304 338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:32.029104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:40881)
0809 19:23:32.029222 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:32.029259 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260809 19:23:32.907321 339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:32.907132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:55699)
0809 19:23:32.907243 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:32.907279 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260809 19:23:32.907903 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:32.907730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:60263)
0809 19:23:32.907835 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:32.907868 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260809 19:23:33.029904 345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:33.029706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:33777)
0809 19:23:33.029829 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:33.029863 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260809 19:23:33.719154 348 raft_consensus.cc:482] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:33.719249 348 raft_consensus.cc:504] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:33.719416 348 leader_election.cc:288] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
W20260809 19:23:33.719820 32156 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103): Not found: Tablet not found: d83cf046896748969f5591a535101c21
W20260809 19:23:33.719913 32160 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487): Not found: Tablet not found: d83cf046896748969f5591a535101c21
I20260809 19:23:33.719985 32160 leader_election.cc:302] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb8ebfd206c4242a3a52b9cd1d63846; no voters: 44c51dde9b054c44a26708d22ac22655, e8372f39adb34f1dbb7e6dbac74f836e
I20260809 19:23:33.720077 348 raft_consensus.cc:2726] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:23:33.907892 349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:33.907697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:48875)
0809 19:23:33.907815 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:33.907849 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260809 19:23:33.908416 350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:33.908234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:50853)
0809 19:23:33.908330 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:33.908360 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":46,"threads_started":1}
I20260809 19:23:34.030450 351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:34.030260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:54897)
0809 19:23:34.030366 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:34.030396 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260809 19:23:34.908475 352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:34.908281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:56759)
0809 19:23:34.908397 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:34.908431 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260809 19:23:34.908947 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:34.908760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:37095)
0809 19:23:34.908878 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:34.908908 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260809 19:23:35.031029 354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:35.030832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:51967)
0809 19:23:35.030959 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:35.030988 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260809 19:23:35.909065 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:35.908855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:53907)
0809 19:23:35.908984 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:35.909022 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260809 19:23:35.909458 357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:35.909279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:60133)
0809 19:23:35.909390 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:35.909420 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260809 19:23:36.031620 358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:36.031393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:55825)
0809 19:23:36.031542 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:36.031576 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260809 19:23:36.734871 359 raft_consensus.cc:482] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:36.734983 359 raft_consensus.cc:504] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44c51dde9b054c44a26708d22ac22655" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 34487 } } peers { permanent_uuid: "e8372f39adb34f1dbb7e6dbac74f836e" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37103 } } peers { permanent_uuid: "4cb8ebfd206c4242a3a52b9cd1d63846" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 44001 } }
I20260809 19:23:36.735132 359 leader_election.cc:288] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487), e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103)
W20260809 19:23:36.735462 32156 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8372f39adb34f1dbb7e6dbac74f836e (127.9.91.66:37103): Not found: Tablet not found: d83cf046896748969f5591a535101c21
W20260809 19:23:36.735558 32160 leader_election.cc:341] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44c51dde9b054c44a26708d22ac22655 (127.9.91.67:34487): Not found: Tablet not found: d83cf046896748969f5591a535101c21
I20260809 19:23:36.735616 32160 leader_election.cc:302] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4cb8ebfd206c4242a3a52b9cd1d63846; no voters: 44c51dde9b054c44a26708d22ac22655, e8372f39adb34f1dbb7e6dbac74f836e
I20260809 19:23:36.735692 359 raft_consensus.cc:2726] T d83cf046896748969f5591a535101c21 P 4cb8ebfd206c4242a3a52b9cd1d63846 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:23:36.909772 363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:36.909533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:57401)
0809 19:23:36.909671 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:36.909715 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260809 19:23:36.910082 365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:36.909831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:37701)
0809 19:23:36.910000 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:36.910036 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":55,"threads_started":1}
I20260809 19:23:37.032266 366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:37.032020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:59091)
0809 19:23:37.032164 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:37.032218 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260809 19:23:37.910415 367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:37.910209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:41681)
0809 19:23:37.910340 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:37.910373 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260809 19:23:37.910629 369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:37.910442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:41983)
0809 19:23:37.910553 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:37.910587 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":49,"threads_started":1}
I20260809 19:23:38.032838 373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:38.032647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:44811)
0809 19:23:38.032762 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:38.032798 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260809 19:23:38.911111 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:38.910893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.65:60179)
0809 19:23:38.911037 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:38.911069 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260809 19:23:38.911284 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:38.911084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.66:48995)
0809 19:23:38.911189 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:38.911236 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260809 19:23:39.033447 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:39.033242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36485 (local address 127.9.91.67:35675)
0809 19:23:39.033376 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:39.033408 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260809 19:23:39.035514 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 32083
I20260809 19:23:39.051467 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 32339
I20260809 19:23:39.063517 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 32539
2026-08-09T19:23:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18197 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:23:39 dist-test-slave-1vmg krb5kdc[381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:23:39 dist-test-slave-1vmg krb5kdc[381](info): set up 2 sockets
Aug 09 19:23:39 dist-test-slave-1vmg krb5kdc[381](info): commencing operation
krb5kdc: starting...
W20260809 19:23:40.193747 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:23:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:23:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:23:40.303421 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43455
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:43455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:40.381681 401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:40.381867 401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:40.381898 401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:40.382995 401 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:23:40.383031 401 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:23:40.383049 401 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:23:40.383062 401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:23:40.383080 401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:40.383097 401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:40.383121 401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:23:40.383138 401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:23:40.384569 401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:43455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:40.384790 401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:40.384979 401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:40.389406 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:40.389436 411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:40.389436 413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:40.389495 401 server_base.cc:1034] running on GCE node
I20260809 19:23:40.389859 401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:40.390122 401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:40.391255 401 hybrid_clock.cc:648] HybridClock initialized: now 1786303420391242 us; error 30 us; skew 500 ppm
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:40.393497 401 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:23:40.393908 401 webserver.cc:460] Webserver started at http://127.9.91.126:45229/ using document root <none> and password file <none>
I20260809 19:23:40.394090 401 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:40.394136 401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:40.394259 401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:40.395238 401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "88945e2ec8754344b1b9269584d3f187"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "1e9237108d350278914388ba1f446876"
server_key_iv: "d838ee976c908cfb21d97a14c3c007f4"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.395551 401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "88945e2ec8754344b1b9269584d3f187"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "1e9237108d350278914388ba1f446876"
server_key_iv: "d838ee976c908cfb21d97a14c3c007f4"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.396821 401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:23:40.397475 425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.397624 401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:40.397707 401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "88945e2ec8754344b1b9269584d3f187"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "1e9237108d350278914388ba1f446876"
server_key_iv: "d838ee976c908cfb21d97a14c3c007f4"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.397765 401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:40.416170 401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:40.417917 401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:40.418049 401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:40.422062 401 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:43455
I20260809 19:23:40.422096 505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:43455 every 8 connection(s)
I20260809 19:23:40.422403 401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:23:40.423259 506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:40.425405 506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Bootstrap starting.
I20260809 19:23:40.425981 506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:40.426247 506 log.cc:824] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:40.426855 506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: No bootstrap required, opened a new log
I20260809 19:23:40.427569 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 401
I20260809 19:23:40.427675 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:23:40.427873 9581 external_mini_cluster.cc:1428] Setting key 34b81d3aa71f2852bb69a290356e425c
I20260809 19:23:40.428850 506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:23:40.428992 506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:40.429028 506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88945e2ec8754344b1b9269584d3f187, State: Initialized, Role: FOLLOWER
I20260809 19:23:40.429195 506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:23:40.429279 506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:23:40.429318 506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:23:40.429363 506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:40.430207 506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:23:40.430294 506 leader_election.cc:302] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88945e2ec8754344b1b9269584d3f187; no voters:
I20260809 19:23:40.430440 506 leader_election.cc:288] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:23:40.430492 512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:40.430635 512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 LEADER]: Becoming Leader. State: Replica: 88945e2ec8754344b1b9269584d3f187, State: Running, Role: LEADER
I20260809 19:23:40.430682 506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:23:40.430747 512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:23:40.431097 512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88945e2ec8754344b1b9269584d3f187. Latest consensus state: current_term: 1 leader_uuid: "88945e2ec8754344b1b9269584d3f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } } }
I20260809 19:23:40.431324 512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: This master's current role is: LEADER
I20260809 19:23:40.431258 514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88945e2ec8754344b1b9269584d3f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } } }
I20260809 19:23:40.431713 514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: This master's current role is: LEADER
W20260809 19:23:40.432449 541 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:23:40.432541 541 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:23:40.432639 528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:23:40.432827 528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:40.434679 528 catalog_manager.cc:1347] Generated new cluster ID: b3ecaf43a48c4536b1eaad3fa9521bde
I20260809 19:23:40.434718 528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:23:40.438437 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.428374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.9.91.126:43455)
0809 19:23:40.428532 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.428541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.428565 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:23:40.428820 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.428825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.428866 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.428967 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:40.429726 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.430818 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.431900 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.432440 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.432545 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.435170 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:40.435174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:40.435174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:40.435185 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:40.436945 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.437338 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.437340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.437341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.437445 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.437626 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.437626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.437627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.437881 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:40.438019 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.438135 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.438255 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260809 19:23:40.463500 528 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:23:40.463974 528 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:23:40.470048 528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Generated new TSK 0
I20260809 19:23:40.470191 528 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:23:40.501091 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43455
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:40.581828 550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:40.582001 550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:40.582022 550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:40.583096 550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:40.583128 550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:40.583153 550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:23:40.584520 550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:40.584683 550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:40.584852 550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:40.589567 559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:40.589602 557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:40.589718 550 server_base.cc:1034] running on GCE node
W20260809 19:23:40.589820 556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:40.589994 550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:40.590217 550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:40.591355 550 hybrid_clock.cc:648] HybridClock initialized: now 1786303420591341 us; error 28 us; skew 500 ppm
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:40.593597 550 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:23:40.593979 550 webserver.cc:460] Webserver started at http://127.9.91.65:34355/ using document root <none> and password file <none>
I20260809 19:23:40.594182 550 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:40.594228 550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:40.594347 550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:40.595319 550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "0ae8df6469134ce1916421235be68f2b"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "466c5624aa736979023e7913b0c574f1"
server_key_iv: "0f2a5c086ee56dd3b1bbb6aadb80e8ea"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.595633 550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "0ae8df6469134ce1916421235be68f2b"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "466c5624aa736979023e7913b0c574f1"
server_key_iv: "0f2a5c086ee56dd3b1bbb6aadb80e8ea"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.596776 550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:23:40.597390 574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.597532 550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:23:40.597610 550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "0ae8df6469134ce1916421235be68f2b"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "466c5624aa736979023e7913b0c574f1"
server_key_iv: "0f2a5c086ee56dd3b1bbb6aadb80e8ea"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.597675 550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:40.613631 550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:40.615244 550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:40.615370 550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:40.615557 550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:40.615825 550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:40.615859 550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.615885 550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:40.615902 550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.621508 550 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:35821
I20260809 19:23:40.621558 752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:35821 every 8 connection(s)
I20260809 19:23:40.621827 550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:23:40.625624 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 550
I20260809 19:23:40.625730 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:23:40.625921 9581 external_mini_cluster.cc:1428] Setting key 6c467c0e80594353281453399aef5edb
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:40.630618 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.622455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:56621 (local address 127.9.91.126:43455)
0809 19:23:40.622502 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.622504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.622678 (+ 174us) server_negotiation.cc:407] Connection header received
0809 19:23:40.623146 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.623147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.623158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.623194 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:40.623989 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.624466 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.625366 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.625657 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.625751 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.627975 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:40.627977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:40.627977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:40.627985 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:40.629401 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.629766 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.629767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.629768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.629833 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.630030 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.630031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.630032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.630170 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:40.630232 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.630439 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.630532 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:23:40.630769 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.622419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:56621)
0809 19:23:40.622561 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.622597 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.623056 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.623247 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.623250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.623749 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:40.623921 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.623925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.624522 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.624522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.625274 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.625276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.625759 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.625760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.625879 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.626316 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:40.626324 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:40.627888 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:40.629476 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:40.629483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:40.629484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:40.629693 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:40.629891 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:40.629892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:40.629892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:40.629979 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:40.630229 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:40.630231 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:40.630332 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 19:23:40.630402 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 19:23:40.630528 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260809 19:23:40.631234 753 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43455
I20260809 19:23:40.631315 753 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:40.631487 753 heartbeater.cc:511] Master 127.9.91.126:43455 requested a full tablet report, sending...
I20260809 19:23:40.632062 451 ts_manager.cc:194] Registered new tserver with Master: 0ae8df6469134ce1916421235be68f2b (127.9.91.65:35821)
I20260809 19:23:40.632988 451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:56621
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:23:40.651463 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43455
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:40.730562 764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:40.730822 764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:40.730854 764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:40.732085 764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:40.732131 764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:40.732158 764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:23:40.733546 764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:40.733718 764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:40.733896 764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:40.738564 780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:40.738587 777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:40.738556 778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:40.738703 764 server_base.cc:1034] running on GCE node
I20260809 19:23:40.738948 764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:40.739180 764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:40.740319 764 hybrid_clock.cc:648] HybridClock initialized: now 1786303420740307 us; error 29 us; skew 500 ppm
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:40.742579 764 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:23:40.743043 764 webserver.cc:460] Webserver started at http://127.9.91.66:34039/ using document root <none> and password file <none>
I20260809 19:23:40.743239 764 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:40.743286 764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:40.743412 764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:40.744354 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "b9c085b6ba53406abf487163fcb692dc"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "ff15284edc24b39345abb4eedf827138"
server_key_iv: "0ccfee673ab4d76797f668e6f9674856"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.744673 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "b9c085b6ba53406abf487163fcb692dc"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "ff15284edc24b39345abb4eedf827138"
server_key_iv: "0ccfee673ab4d76797f668e6f9674856"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.745831 764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:23:40.746440 796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.746593 764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:23:40.746683 764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "b9c085b6ba53406abf487163fcb692dc"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "ff15284edc24b39345abb4eedf827138"
server_key_iv: "0ccfee673ab4d76797f668e6f9674856"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.746747 764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:40.754551 764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:40.756119 764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:40.756245 764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:40.756434 764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:40.756706 764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:40.756740 764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.756767 764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:40.756783 764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.762303 764 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:33473
I20260809 19:23:40.762338 972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:33473 every 8 connection(s)
I20260809 19:23:40.762593 764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:23:40.765560 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 764
I20260809 19:23:40.765651 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:23:40.765831 9581 external_mini_cluster.cc:1428] Setting key d53f0264f60e99b96f819ec4f5a85b12
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:40.771206 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.763237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:33039 (local address 127.9.91.126:43455)
0809 19:23:40.763311 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.763313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.763444 (+ 131us) server_negotiation.cc:407] Connection header received
0809 19:23:40.763846 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.763847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.763856 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.763900 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:40.764705 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.765191 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.766197 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.766402 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.766462 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.768710 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:40.768711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:40.768711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:40.768718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:40.769855 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.770279 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.770280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.770280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.770327 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.770604 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.770605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.770605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.770703 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:40.770746 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.771071 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.771143 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:23:40.771417 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.763200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:33039)
0809 19:23:40.763344 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.763377 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.763778 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.763952 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.763954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.764455 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:40.764641 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.764645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.765249 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.765249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.766130 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.766133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.766491 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.766493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.766640 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.767092 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:40.767101 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:40.768622 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:40.769930 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:40.769938 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:40.769940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:40.770210 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:40.770374 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:40.770375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:40.770375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:40.770517 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:40.770759 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:40.770767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:40.770940 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 19:23:40.771053 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 19:23:40.771212 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260809 19:23:40.771854 978 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43455
I20260809 19:23:40.771938 978 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:40.772089 978 heartbeater.cc:511] Master 127.9.91.126:43455 requested a full tablet report, sending...
I20260809 19:23:40.772440 451 ts_manager.cc:194] Registered new tserver with Master: b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473)
I20260809 19:23:40.772799 451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:33039
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:23:40.789738 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43455
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:23:40.869230 985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:23:40.869414 985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:23:40.869449 985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:23:40.870539 985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:23:40.870585 985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:23:40.870623 985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:23:40.872069 985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:23:40.872310 985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:23:40.872519 985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:23:40.877102 998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:40.877104 995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:23:40.877194 994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:23:40.877300 985 server_base.cc:1034] running on GCE node
I20260809 19:23:40.877490 985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:23:40.877704 985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:23:40.878854 985 hybrid_clock.cc:648] HybridClock initialized: now 1786303420878838 us; error 34 us; skew 500 ppm
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:23:40.881256 985 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:23:40.881682 985 webserver.cc:460] Webserver started at http://127.9.91.67:41013/ using document root <none> and password file <none>
I20260809 19:23:40.881878 985 fs_manager.cc:359] Metadata directory not provided
I20260809 19:23:40.881929 985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:23:40.882066 985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:23:40.883085 985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "6cd2119e0b9d412cad61fe3df5d82b7c"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "504f28d9ed415bcb466838ae66396d8c"
server_key_iv: "0b0a432f4f1f9c68ba3fa240b8202dfb"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.883416 985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "6cd2119e0b9d412cad61fe3df5d82b7c"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "504f28d9ed415bcb466838ae66396d8c"
server_key_iv: "0b0a432f4f1f9c68ba3fa240b8202dfb"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.884678 985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:23:40.885318 1006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.885468 985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.885556 985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "6cd2119e0b9d412cad61fe3df5d82b7c"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "504f28d9ed415bcb466838ae66396d8c"
server_key_iv: "0b0a432f4f1f9c68ba3fa240b8202dfb"
server_key_version: "encryptionkey@0"
I20260809 19:23:40.885619 985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:23:40.913050 985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:23:40.915155 985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:23:40.915295 985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:23:40.915509 985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:23:40.915797 985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:23:40.915836 985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.915866 985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:23:40.915894 985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:23:40.922021 985 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:40033
I20260809 19:23:40.922056 1170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:40033 every 8 connection(s)
I20260809 19:23:40.922335 985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:23:40.923736 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 985
I20260809 19:23:40.923843 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:23:40.924039 9581 external_mini_cluster.cc:1428] Setting key 7a6502f3c76b71e16c4212844c1347a6
Aug 09 19:23:40 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303420, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:23:40.930612 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.923002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:58141 (local address 127.9.91.126:43455)
0809 19:23:40.923052 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.923055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.923240 (+ 185us) server_negotiation.cc:407] Connection header received
0809 19:23:40.923718 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.923720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.923733 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.923780 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:40.924795 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.925318 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.926252 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.926425 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.926468 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.928415 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:40.928416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:40.928416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:40.928423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:40.929473 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.929813 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.929813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.929814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.929856 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.930071 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.930071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.930072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.930212 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:40.930271 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.930492 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.930550 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:23:40.930774 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.922979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:58141)
0809 19:23:40.923132 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.923164 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.923612 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.923844 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.923847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.924466 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:23:40.924677 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.924683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.925396 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.925397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.926183 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.926185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.926479 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.926480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.926577 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.927085 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:23:40.927093 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:23:40.928339 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:23:40.929538 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:40.929544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:40.929545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:40.929753 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:40.929902 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:23:40.929902 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:23:40.929903 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:23:40.930007 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:23:40.930272 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:23:40.930278 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:23:40.930386 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 19:23:40.930467 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 19:23:40.930586 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260809 19:23:40.931258 1172 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43455
I20260809 19:23:40.931352 1172 heartbeater.cc:464] Registering TS with master...
I20260809 19:23:40.931510 1172 heartbeater.cc:511] Master 127.9.91.126:43455 requested a full tablet report, sending...
I20260809 19:23:40.931856 451 ts_manager.cc:194] Registered new tserver with Master: 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
I20260809 19:23:40.932293 451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:58141
I20260809 19:23:40.935655 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:23:40.942078 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.936888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.9.91.126:43455)
0809 19:23:40.936933 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.936935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.937040 (+ 105us) server_negotiation.cc:407] Connection header received
0809 19:23:40.937099 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.937100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.937109 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.937152 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:23:40.937672 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.938107 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.938790 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.939037 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.939086 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.940073 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:23:40.940075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:23:40.940075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:23:40.940081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:23:40.941106 (+ 1025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.941397 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.941398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.941398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.941449 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:23:40.941608 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:23:40.941608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:23:40.941609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:23:40.941741 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:23:40.941822 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.941930 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.942000 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:23:40.943387 451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:23:40.944876 451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:23:40.950722 1187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.946689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:33473 (local address 127.0.0.1:53356)
0809 19:23:40.947080 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.947089 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.947142 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.947475 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.947476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.947477 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:40.947597 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.947599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.948472 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.948473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.949718 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.949721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.950437 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.950438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.950514 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.950527 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:23:40.950590 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 19:23:40.950643 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":15,"thread_start_us":68,"threads_started":1}
I20260809 19:23:40.950733 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.946815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53356 (local address 127.9.91.66:33473)
0809 19:23:40.947343 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.947348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.947357 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:23:40.947395 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.947397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.947412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.947480 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:40.947682 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.948382 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.949810 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.950365 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.950427 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.950484 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.950611 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.950652 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":130,"threads_started":1}
I20260809 19:23:40.951025 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.946863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40033 (local address 127.0.0.1:52008)
0809 19:23:40.947156 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.947175 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.947222 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.947442 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.947443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.947446 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:40.947592 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.947595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.948796 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.948798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.949707 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.949710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.950774 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.950775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.950830 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.950834 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:23:40.950881 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:23:40.950935 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":45,"threads_started":1}
I20260809 19:23:40.951041 1185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.946688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35821 (local address 127.0.0.1:59748)
0809 19:23:40.946825 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.946864 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.946957 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.947277 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.947280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.947282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:40.947342 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.947344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.948745 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.948746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.949816 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.949818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.950762 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.950763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:40.950837 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.950840 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:23:40.950867 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 19:23:40.950934 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":42,"threads_started":1}
I20260809 19:23:40.951221 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.946880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.9.91.65:35821)
0809 19:23:40.947106 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.947111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.947123 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:23:40.947167 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.947169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.947186 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.947273 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:40.947419 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.948638 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.949896 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.950670 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.950979 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.951068 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.951102 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.951148 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
I20260809 19:23:40.951452 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.947026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52008 (local address 127.9.91.67:40033)
0809 19:23:40.947181 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.947184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.947220 (+ 36us) server_negotiation.cc:407] Connection header received
0809 19:23:40.947277 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.947279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.947293 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.947368 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:40.947685 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.948712 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.949788 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.950610 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.950686 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:40.950745 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:40.951363 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 19:23:40.951397 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260809 19:23:40.951673 869 tablet_service.cc:1467] Processing CreateTablet for tablet b0ad5de4bc3d4f20940a620e378ce0ef (DEFAULT_TABLE table=test-table [id=ade9acf3795f48e48516b6fc2ab019de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:40.952018 869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ad5de4bc3d4f20940a620e378ce0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:40.952062 1086 tablet_service.cc:1467] Processing CreateTablet for tablet b0ad5de4bc3d4f20940a620e378ce0ef (DEFAULT_TABLE table=test-table [id=ade9acf3795f48e48516b6fc2ab019de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:40.952070 648 tablet_service.cc:1467] Processing CreateTablet for tablet b0ad5de4bc3d4f20940a620e378ce0ef (DEFAULT_TABLE table=test-table [id=ade9acf3795f48e48516b6fc2ab019de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:23:40.952381 648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ad5de4bc3d4f20940a620e378ce0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:40.952378 1086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0ad5de4bc3d4f20940a620e378ce0ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:23:40.954455 1194 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Bootstrap starting.
I20260809 19:23:40.955147 1194 tablet_bootstrap.cc:654] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:40.955529 1195 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Bootstrap starting.
I20260809 19:23:40.955595 1197 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Bootstrap starting.
I20260809 19:23:40.955806 1194 log.cc:824] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:40.956207 1195 tablet_bootstrap.cc:654] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:40.956274 1197 tablet_bootstrap.cc:654] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Neither blocks nor log segments found. Creating new log.
I20260809 19:23:40.956624 1194 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: No bootstrap required, opened a new log
I20260809 19:23:40.956650 1195 log.cc:824] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:40.956703 1194 ts_tablet_manager.cc:1397] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:23:40.956868 1197 log.cc:824] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:23:40.957371 1195 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: No bootstrap required, opened a new log
I20260809 19:23:40.957432 1195 ts_tablet_manager.cc:1397] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:23:40.957631 1197 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: No bootstrap required, opened a new log
I20260809 19:23:40.957738 1197 ts_tablet_manager.cc:1397] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:23:40.958575 1194 raft_consensus.cc:348] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:40.958765 1194 raft_consensus.cc:374] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:40.958802 1194 raft_consensus.cc:729] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c085b6ba53406abf487163fcb692dc, State: Initialized, Role: FOLLOWER
I20260809 19:23:40.958972 1194 consensus_queue.cc:260] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:40.959048 1195 raft_consensus.cc:348] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:40.959218 1195 raft_consensus.cc:374] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:40.959260 1195 raft_consensus.cc:729] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae8df6469134ce1916421235be68f2b, State: Initialized, Role: FOLLOWER
I20260809 19:23:40.959259 1194 ts_tablet_manager.cc:1428] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 19:23:40.959388 978 heartbeater.cc:503] Master 127.9.91.126:43455 was elected leader, sending a full tablet report...
I20260809 19:23:40.959373 1195 consensus_queue.cc:260] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:40.959636 1195 ts_tablet_manager.cc:1428] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:23:40.959676 753 heartbeater.cc:503] Master 127.9.91.126:43455 was elected leader, sending a full tablet report...
I20260809 19:23:40.959658 1197 raft_consensus.cc:348] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:40.959837 1197 raft_consensus.cc:374] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:23:40.959908 1197 raft_consensus.cc:729] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd2119e0b9d412cad61fe3df5d82b7c, State: Initialized, Role: FOLLOWER
I20260809 19:23:40.960059 1197 consensus_queue.cc:260] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:40.960325 1197 ts_tablet_manager.cc:1428] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 19:23:40.960450 1172 heartbeater.cc:503] Master 127.9.91.126:43455 was elected leader, sending a full tablet report...
I20260809 19:23:40.997143 1204 raft_consensus.cc:482] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:23:40.997255 1204 raft_consensus.cc:504] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:40.997565 1204 leader_election.cc:288] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae8df6469134ce1916421235be68f2b (127.9.91.65:35821), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
I20260809 19:23:41.001185 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.997724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35821 (local address 127.9.91.66:59187)
0809 19:23:40.997780 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.997791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.997830 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.997956 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.997957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.997958 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:40.998024 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.998026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.998893 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.998894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:41.000123 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:41.000126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:41.000909 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.000909 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:41.001026 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:41.001031 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:23:41.001079 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 19:23:41.001118 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":3}
I20260809 19:23:41.001209 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.997772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:59187 (local address 127.9.91.65:35821)
0809 19:23:40.997820 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.997822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.997832 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:23:40.997887 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.997888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.997898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.997956 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:40.998086 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.998818 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.000209 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:41.000817 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.000893 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:41.000927 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:41.001093 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 19:23:41.001137 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:23:41.001291 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.997739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40033 (local address 127.9.91.66:38901)
0809 19:23:40.997892 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:40.997901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:23:40.997954 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:23:40.998090 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:23:40.998091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:23:40.998093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:23:40.998148 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:40.998149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.998992 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:40.998993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:41.000123 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:23:41.000126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:23:41.000908 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.000909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:23:41.001100 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:41.001106 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 19:23:41.001172 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:23:41.001213 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":73,"thread_start_us":104,"threads_started":1}
I20260809 19:23:41.001299 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:40.997794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:38901 (local address 127.9.91.67:40033)
0809 19:23:40.997831 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:40.997833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:40.997965 (+ 132us) server_negotiation.cc:407] Connection header received
0809 19:23:40.998013 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:40.998015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:40.998026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:40.998082 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:23:40.998209 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:40.998888 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.000216 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:41.000817 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.000897 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:41.000905 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:41.001199 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 19:23:41.001243 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:23:41.001470 682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad5de4bc3d4f20940a620e378ce0ef" candidate_uuid: "b9c085b6ba53406abf487163fcb692dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae8df6469134ce1916421235be68f2b" is_pre_election: true
I20260809 19:23:41.001605 1115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad5de4bc3d4f20940a620e378ce0ef" candidate_uuid: "b9c085b6ba53406abf487163fcb692dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" is_pre_election: true
I20260809 19:23:41.001650 682 raft_consensus.cc:2454] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9c085b6ba53406abf487163fcb692dc in term 0.
I20260809 19:23:41.001732 1115 raft_consensus.cc:2454] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9c085b6ba53406abf487163fcb692dc in term 0.
I20260809 19:23:41.001878 799 leader_election.cc:302] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b, b9c085b6ba53406abf487163fcb692dc; no voters:
I20260809 19:23:41.001981 1204 raft_consensus.cc:2781] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:23:41.002048 1204 raft_consensus.cc:482] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:23:41.002072 1204 raft_consensus.cc:3037] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:41.002782 1204 raft_consensus.cc:504] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:41.002902 1204 leader_election.cc:288] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [CANDIDATE]: Term 1 election: Requested vote from peers 0ae8df6469134ce1916421235be68f2b (127.9.91.65:35821), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
I20260809 19:23:41.003072 682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad5de4bc3d4f20940a620e378ce0ef" candidate_uuid: "b9c085b6ba53406abf487163fcb692dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae8df6469134ce1916421235be68f2b"
I20260809 19:23:41.003101 1115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0ad5de4bc3d4f20940a620e378ce0ef" candidate_uuid: "b9c085b6ba53406abf487163fcb692dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c"
I20260809 19:23:41.003142 682 raft_consensus.cc:3037] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:41.003172 1115 raft_consensus.cc:3037] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:23:41.003829 682 raft_consensus.cc:2454] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9c085b6ba53406abf487163fcb692dc in term 1.
I20260809 19:23:41.003835 1115 raft_consensus.cc:2454] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9c085b6ba53406abf487163fcb692dc in term 1.
I20260809 19:23:41.004016 802 leader_election.cc:302] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6cd2119e0b9d412cad61fe3df5d82b7c, b9c085b6ba53406abf487163fcb692dc; no voters:
I20260809 19:23:41.004112 1204 raft_consensus.cc:2781] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:23:41.004242 1204 raft_consensus.cc:686] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 1 LEADER]: Becoming Leader. State: Replica: b9c085b6ba53406abf487163fcb692dc, State: Running, Role: LEADER
I20260809 19:23:41.004319 1204 consensus_queue.cc:237] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:23:41.005121 448 catalog_manager.cc:5495] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc reported cstate change: term changed from 0 to 1, leader changed from <none> to b9c085b6ba53406abf487163fcb692dc (127.9.91.66). New cstate: current_term: 1 leader_uuid: "b9c085b6ba53406abf487163fcb692dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } health_report { overall_health: UNKNOWN } } }
I20260809 19:23:41.010421 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:41.008401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.9.91.66:33473)
0809 19:23:41.008446 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:23:41.008448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:23:41.008456 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:23:41.008518 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:23:41.008519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:23:41.008536 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:23:41.008597 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:23:41.008674 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:41.009106 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.009831 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:23:41.010025 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:23:41.010082 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:23:41.010172 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:23:41.010255 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:23:41.010310 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:23:41.010325 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 19:23:41.010341 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:23:41.011973 1115 raft_consensus.cc:1261] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Refusing update from remote peer b9c085b6ba53406abf487163fcb692dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:41.012012 682 raft_consensus.cc:1261] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Refusing update from remote peer b9c085b6ba53406abf487163fcb692dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:23:41.012317 1204 consensus_queue.cc:1035] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:23:41.012440 1210 consensus_queue.cc:1035] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:23:41.013111 980 tablet.cc:2369] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:23:41.015262 1215 mvcc.cc:205] Tried to move back new op lower bound from 7316698812462956544 to 7316698812434378752. Current Snapshot: MvccSnapshot[applied={T|T < 7316698812462956544 or (T in {7316698812462956544})}]
I20260809 19:23:41.015724 1216 mvcc.cc:205] Tried to move back new op lower bound from 7316698812462956544 to 7316698812434378752. Current Snapshot: MvccSnapshot[applied={T|T < 7316698812462956544}]
I20260809 19:23:41.016984 448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:41.017078 448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:23:41.017751 448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Sending DeleteTablet for 3 replicas of tablet b0ad5de4bc3d4f20940a620e378ce0ef
I20260809 19:23:41.018059 869 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad5de4bc3d4f20940a620e378ce0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:41 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53356
I20260809 19:23:41.018086 648 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad5de4bc3d4f20940a620e378ce0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:41 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:59748
I20260809 19:23:41.018283 1086 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad5de4bc3d4f20940a620e378ce0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:41 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:52008
I20260809 19:23:41.018316 1233 tablet_replica.cc:330] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: stopping tablet replica
I20260809 19:23:41.018448 1233 raft_consensus.cc:2229] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:41.018507 1235 tablet_replica.cc:330] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: stopping tablet replica
I20260809 19:23:41.018559 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 550
I20260809 19:23:41.018622 1233 raft_consensus.cc:2258] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:41.018657 1235 raft_consensus.cc:2229] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:23:41.018756 1235 raft_consensus.cc:2258] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:41.024008 1235 ts_tablet_manager.cc:1905] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:41.025606 1235 ts_tablet_manager.cc:1918] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:41.025671 1235 log.cc:1196] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/b0ad5de4bc3d4f20940a620e378ce0ef
I20260809 19:23:41.025926 1235 ts_tablet_manager.cc:1939] T b0ad5de4bc3d4f20940a620e378ce0ef P 6cd2119e0b9d412cad61fe3df5d82b7c: Deleting consensus metadata
I20260809 19:23:41.031263 433 catalog_manager.cc:4841] TS 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033): tablet b0ad5de4bc3d4f20940a620e378ce0ef (table test-table [id=ade9acf3795f48e48516b6fc2ab019de]) successfully deleted
I20260809 19:23:41.031658 1231 tablet_replica.cc:330] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: stopping tablet replica
I20260809 19:23:41.031790 1231 raft_consensus.cc:2229] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:23:41.031890 1231 raft_consensus.cc:2258] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:23:41.032271 1231 ts_tablet_manager.cc:1905] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:23:41.033494 1231 ts_tablet_manager.cc:1918] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:23:41.033569 1231 log.cc:1196] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/b0ad5de4bc3d4f20940a620e378ce0ef
I20260809 19:23:41.033818 1231 ts_tablet_manager.cc:1939] T b0ad5de4bc3d4f20940a620e378ce0ef P b9c085b6ba53406abf487163fcb692dc: Deleting consensus metadata
I20260809 19:23:41.034142 433 catalog_manager.cc:4841] TS b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473): tablet b0ad5de4bc3d4f20940a620e378ce0ef (table test-table [id=ade9acf3795f48e48516b6fc2ab019de]) successfully deleted
W20260809 19:23:41.036420 428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:35821 (error 108)
I20260809 19:23:41.036671 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 764
I20260809 19:23:41.036902 1185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:23:41.036797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35821 (local address 127.0.0.1:59764)
0809 19:23:41.036837 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:23:41.036872 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 19:23:41.038352 428 catalog_manager.cc:4584] TS 0ae8df6469134ce1916421235be68f2b (127.9.91.65:35821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0ad5de4bc3d4f20940a620e378ce0ef: Network error: Client connection negotiation failed: client connection to 127.9.91.65:35821: connect: Connection refused (error 111)
I20260809 19:23:41.044253 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 985
I20260809 19:23:41.068279 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 401
I20260809 19:24:01.080344 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43455
--webserver_interface=127.9.91.126
--webserver_port=45229
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:43455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:01.160808 1237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:01.160990 1237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:01.161011 1237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:01.162076 1237 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:24:01.162108 1237 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:24:01.162120 1237 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:24:01.162130 1237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:01.162142 1237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:01.162158 1237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:01.162169 1237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:01.162185 1237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:01.163586 1237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:43455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=45229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:01.163772 1237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:01.163936 1237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:01.172569 1249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:01.172577 1246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:01.172683 1247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:01.172737 1237 server_base.cc:1034] running on GCE node
I20260809 19:24:01.173036 1237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:01.173333 1237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:01.174479 1237 hybrid_clock.cc:648] HybridClock initialized: now 1786303441174459 us; error 35 us; skew 500 ppm
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:01.179811 1237 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:24:01.180205 1237 webserver.cc:460] Webserver started at http://127.9.91.126:45229/ using document root <none> and password file <none>
I20260809 19:24:01.180402 1237 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:01.180466 1237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:01.181716 1237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 19:24:01.182287 1259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.182447 1237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.182538 1237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "88945e2ec8754344b1b9269584d3f187"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "1e9237108d350278914388ba1f446876"
server_key_iv: "d838ee976c908cfb21d97a14c3c007f4"
server_key_version: "encryptionkey@0"
I20260809 19:24:01.182870 1237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:01.204399 1237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:01.205988 1237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:01.206116 1237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:01.209921 1237 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:43455
I20260809 19:24:01.209952 1343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:43455 every 8 connection(s)
I20260809 19:24:01.210294 1237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:01.211697 1345 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:24:01.212291 1345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Bootstrap starting.
I20260809 19:24:01.216159 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 1237
I20260809 19:24:01.216400 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:35821
--local_ip_for_outbound_sockets=127.9.91.65
--tserver_master_addrs=127.9.91.126:43455
--webserver_port=34355
--webserver_interface=127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:24:01.221519 1345 log.cc:824] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:01.223361 1345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:24:01.223613 1345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Bootstrap complete.
I20260809 19:24:01.225664 1345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:01.225875 1345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88945e2ec8754344b1b9269584d3f187, State: Initialized, Role: FOLLOWER
I20260809 19:24:01.226007 1345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:01.226073 1345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:01.226104 1345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:01.226150 1345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 1 FOLLOWER]: Advancing to term 2
I20260809 19:24:01.227126 1345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:01.227224 1345 leader_election.cc:302] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88945e2ec8754344b1b9269584d3f187; no voters:
I20260809 19:24:01.227375 1345 leader_election.cc:288] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 19:24:01.227437 1351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 19:24:01.227556 1351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 LEADER]: Becoming Leader. State: Replica: 88945e2ec8754344b1b9269584d3f187, State: Running, Role: LEADER
I20260809 19:24:01.227627 1345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:01.227711 1351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:01.227991 1354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "88945e2ec8754344b1b9269584d3f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } } }
I20260809 19:24:01.228025 1351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88945e2ec8754344b1b9269584d3f187. Latest consensus state: current_term: 2 leader_uuid: "88945e2ec8754344b1b9269584d3f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } } }
I20260809 19:24:01.228142 1354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:01.228153 1351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:01.228742 1365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:01.229557 1365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:01.229656 1365 catalog_manager.cc:1259] Loaded cluster ID: b3ecaf43a48c4536b1eaad3fa9521bde
I20260809 19:24:01.229687 1365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:01.230937 1365 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:01.231112 1365 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Loaded TSK: 0
I20260809 19:24:01.249763 1365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Generated new TSK 1
I20260809 19:24:01.249845 1365 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 19:24:01.303284 1347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:01.303463 1347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:01.303486 1347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:01.304593 1347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:01.304632 1347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:01.304656 1347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:01.306043 1347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:35821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=34355
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:01.306248 1347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:01.306461 1347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:01.311123 1387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:01.311141 1390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:01.311228 1347 server_base.cc:1034] running on GCE node
W20260809 19:24:01.311141 1392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:01.311553 1347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:01.311854 1347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:01.313045 1347 hybrid_clock.cc:648] HybridClock initialized: now 1786303441313024 us; error 37 us; skew 500 ppm
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:01.316010 1347 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:24:01.316527 1347 webserver.cc:460] Webserver started at http://127.9.91.65:34355/ using document root <none> and password file <none>
I20260809 19:24:01.316762 1347 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:01.316849 1347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:01.318506 1347 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:24:01.319283 1400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.319464 1347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:01.319558 1347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "0ae8df6469134ce1916421235be68f2b"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "466c5624aa736979023e7913b0c574f1"
server_key_iv: "0f2a5c086ee56dd3b1bbb6aadb80e8ea"
server_key_version: "encryptionkey@0"
I20260809 19:24:01.319864 1347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:01.331676 1347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:01.334044 1347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:01.334196 1347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:01.334391 1347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:01.339102 1415 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:24:01.340809 1347 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:24:01.340924 1347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 19:24:01.341017 1347 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:24:01.341915 1347 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:24:01.342015 1415 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Bootstrap starting.
I20260809 19:24:01.342013 1347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:01.349586 1347 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:35821
I20260809 19:24:01.349924 1347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:24:01.351359 1578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:35821 every 8 connection(s)
I20260809 19:24:01.352654 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 1347
I20260809 19:24:01.352777 1415 log.cc:824] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:01.352953 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:33473
--local_ip_for_outbound_sockets=127.9.91.66
--tserver_master_addrs=127.9.91.126:43455
--webserver_port=34039
--webserver_interface=127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:24:01.357466 1415 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:24:01.357795 1415 tablet_bootstrap.cc:492] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Bootstrap complete.
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:01.358989 1415 ts_tablet_manager.cc:1397] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.014s
I20260809 19:24:01.360014 1415 raft_consensus.cc:348] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:24:01.360294 1415 raft_consensus.cc:729] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae8df6469134ce1916421235be68f2b, State: Initialized, Role: FOLLOWER
I20260809 19:24:01.360455 1415 consensus_queue.cc:260] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } }
I20260809 19:24:01.360728 1415 ts_tablet_manager.cc:1428] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 19:24:01.362093 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.350618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:43965 (local address 127.9.91.126:43455)
0809 19:24:01.352103 (+ 1485us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.352110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.352138 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:24:01.352231 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.352236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.352273 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.352380 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:01.353188 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.354382 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.355697 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.356078 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.356173 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.358425 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:01.358429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:01.358429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:01.358442 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:01.360402 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.360886 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.360889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.360889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.360983 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.361264 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.361265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.361265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.361464 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:01.361524 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.361858 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.361961 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1442,"thread_start_us":105,"threads_started":1}
I20260809 19:24:01.362174 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.350549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:43965)
0809 19:24:01.351507 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.351547 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.351938 (+ 391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.352438 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.352442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.352792 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:01.353055 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.353063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.354532 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.354534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.355603 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.355605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.356182 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.356184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.356320 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.356834 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:01.356845 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:01.358320 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:01.360497 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.360505 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.360507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.360810 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.361062 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.361063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.361063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.361197 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.361541 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:01.361547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:01.361706 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.361821 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.361984 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"thread_start_us":90,"threads_started":1}
I20260809 19:24:01.362865 1582 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43455
I20260809 19:24:01.362979 1582 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:01.363159 1582 heartbeater.cc:511] Master 127.9.91.126:43455 requested a full tablet report, sending...
I20260809 19:24:01.363904 1283 ts_manager.cc:194] Registered new tserver with Master: 0ae8df6469134ce1916421235be68f2b (127.9.91.65:35821)
I20260809 19:24:01.364791 1283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:43965
I20260809 19:24:01.365777 1582 heartbeater.cc:503] Master 127.9.91.126:43455 was elected leader, sending a full tablet report...
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.65@KRBTEST.COM
I20260809 19:24:01.372177 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.364453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35821 (local address 127.0.0.1:50770)
0809 19:24:01.364642 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.364676 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.364740 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.365334 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.365341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.365569 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:01.365609 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.365611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.366587 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.366588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.367168 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.367171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.367601 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.367603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.367675 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.368059 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:01.368064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:01.369403 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:01.371239 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.371245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.371247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.371427 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.371625 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.371626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.371626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.371681 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.371941 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:01.371945 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:01.371981 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.372034 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.372115 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260809 19:24:01.372265 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.365046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.9.91.65:35821)
0809 19:24:01.365203 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.365208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.365216 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:01.365245 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.365247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.365262 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.365328 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:01.365692 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.366508 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.367245 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.367527 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.367585 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.369486 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:01.369494 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:01.369494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:01.369500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:01.371153 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.371500 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.371503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.371503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.371569 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.371735 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.371736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.371736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.371887 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:01.371959 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.372087 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.372177 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260809 19:24:01.372558 1477 tablet_service.cc:1514] Processing DeleteTablet for tablet b0ad5de4bc3d4f20940a620e378ce0ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:23:41 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50770
I20260809 19:24:01.372987 1596 tablet_replica.cc:330] stopping tablet replica
I20260809 19:24:01.373066 1596 raft_consensus.cc:2229] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:01.373123 1596 raft_consensus.cc:2258] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:01.373387 1596 ts_tablet_manager.cc:1905] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:01.375276 1596 ts_tablet_manager.cc:1918] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:01.375335 1596 log.cc:1196] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal/wals/b0ad5de4bc3d4f20940a620e378ce0ef
I20260809 19:24:01.375520 1596 ts_tablet_manager.cc:1939] T b0ad5de4bc3d4f20940a620e378ce0ef P 0ae8df6469134ce1916421235be68f2b: Deleting consensus metadata
I20260809 19:24:01.375932 1261 catalog_manager.cc:4841] TS 0ae8df6469134ce1916421235be68f2b (127.9.91.65:35821): tablet b0ad5de4bc3d4f20940a620e378ce0ef (table test-table [id=ade9acf3795f48e48516b6fc2ab019de]) successfully deleted
W20260809 19:24:01.439066 1592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:01.439245 1592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:01.439276 1592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:01.440394 1592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:01.440433 1592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:01.440469 1592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:01.441854 1592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:33473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=34039
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:01.442102 1592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:01.442302 1592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:01.446861 1610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:01.447021 1603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:01.446892 1604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:01.446972 1592 server_base.cc:1034] running on GCE node
I20260809 19:24:01.447279 1592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:01.447461 1592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:01.448606 1592 hybrid_clock.cc:648] HybridClock initialized: now 1786303441448592 us; error 29 us; skew 500 ppm
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:01.450874 1592 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:24:01.451349 1592 webserver.cc:460] Webserver started at http://127.9.91.66:34039/ using document root <none> and password file <none>
I20260809 19:24:01.451529 1592 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:01.451578 1592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:01.452719 1592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:01.453202 1619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.453393 1592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.453478 1592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "b9c085b6ba53406abf487163fcb692dc"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "ff15284edc24b39345abb4eedf827138"
server_key_iv: "0ccfee673ab4d76797f668e6f9674856"
server_key_version: "encryptionkey@0"
I20260809 19:24:01.453745 1592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:01.464095 1592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:01.466017 1592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:01.466118 1592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:01.466284 1592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:01.466501 1592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:01.466529 1592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.466547 1592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:01.466559 1592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.471800 1592 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:33473
I20260809 19:24:01.471868 1801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:33473 every 8 connection(s)
I20260809 19:24:01.472128 1592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:01.480540 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.472761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:38455 (local address 127.9.91.126:43455)
0809 19:24:01.472821 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.472823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.472981 (+ 158us) server_negotiation.cc:407] Connection header received
0809 19:24:01.473433 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.473435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.473451 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.473501 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:01.474367 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.474861 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.475821 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.476029 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.476086 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.477958 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:01.477959 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:01.477960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:01.477968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:01.479250 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.479616 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.479617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.479618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.479680 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.479922 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.479923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.479923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.480029 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:01.480085 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.480353 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.480446 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:24:01.480647 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.472702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:38455)
0809 19:24:01.472872 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.472906 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.473334 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.473564 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.473567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.474086 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:01.474265 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.474269 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.474968 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.474969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.475728 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.475730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.476108 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.476109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.476227 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.476624 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:01.476632 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:01.477863 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:01.479330 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.479336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.479338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.479541 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.479757 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.479757 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.479758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.479856 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.480095 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:01.480102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:01.480230 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.480317 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.480456 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260809 19:24:01.481081 1802 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43455
I20260809 19:24:01.481159 1802 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:01.481302 1802 heartbeater.cc:511] Master 127.9.91.126:43455 requested a full tablet report, sending...
I20260809 19:24:01.481634 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 1592
I20260809 19:24:01.481665 1283 ts_manager.cc:194] Registered new tserver with Master: b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473)
I20260809 19:24:01.481894 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:40033
--local_ip_for_outbound_sockets=127.9.91.67
--tserver_master_addrs=127.9.91.126:43455
--webserver_port=41013
--webserver_interface=127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:24:01.482052 1283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:38455
W20260809 19:24:01.562180 1808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:01.562361 1808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:01.562383 1808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:01.563544 1808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:01.563591 1808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:01.563623 1808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:01.565053 1808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:40033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=41013
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:01.565274 1808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:01.565500 1808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:01.570115 1816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:01.570098 1817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:01.570089 1820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:01.570331 1808 server_base.cc:1034] running on GCE node
I20260809 19:24:01.570518 1808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:01.570746 1808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:01.571885 1808 hybrid_clock.cc:648] HybridClock initialized: now 1786303441571862 us; error 41 us; skew 500 ppm
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:01.574219 1808 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:24:01.574600 1808 webserver.cc:460] Webserver started at http://127.9.91.67:41013/ using document root <none> and password file <none>
I20260809 19:24:01.574790 1808 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:01.574846 1808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:01.576143 1808 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:01.576699 1833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.576848 1808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:01.576929 1808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "6cd2119e0b9d412cad61fe3df5d82b7c"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "504f28d9ed415bcb466838ae66396d8c"
server_key_iv: "0b0a432f4f1f9c68ba3fa240b8202dfb"
server_key_version: "encryptionkey@0"
I20260809 19:24:01.577209 1808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:01.592778 1808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:01.594368 1808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:01.594511 1808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:01.594715 1808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:01.595044 1808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:01.595085 1808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.595108 1808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:01.595136 1808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.601336 1808 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:40033
I20260809 19:24:01.601370 2014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:40033 every 8 connection(s)
I20260809 19:24:01.601816 1808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:24:01.607082 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 1808
Aug 09 19:24:01 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303441, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:01.610339 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.602473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:60949 (local address 127.9.91.126:43455)
0809 19:24:01.602544 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.602547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.602713 (+ 166us) server_negotiation.cc:407] Connection header received
0809 19:24:01.603205 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.603207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.603217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.603256 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:01.604074 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.604545 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.605449 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.605624 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.605665 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.607763 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:01.607765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:01.607765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:01.607774 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:01.609064 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.609450 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.609451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.609452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.609531 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.609744 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.609745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.609745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.609874 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:01.609946 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.610199 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.610270 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 19:24:01.610502 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.602433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:60949)
0809 19:24:01.602601 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.602633 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.603105 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.603306 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.603309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.603825 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:01.604001 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.604006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.604612 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.604613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.605380 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.605383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.605685 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.605685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.605778 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.606151 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:01.606159 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:01.607657 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:01.609152 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.609159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.609161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.609373 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.609599 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:01.609599 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:01.609600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:01.609686 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:01.609935 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:01.609937 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:01.610067 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.610178 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.610328 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260809 19:24:01.611193 2015 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43455
I20260809 19:24:01.611284 2015 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:01.611452 2015 heartbeater.cc:511] Master 127.9.91.126:43455 requested a full tablet report, sending...
I20260809 19:24:01.611845 1283 ts_manager.cc:194] Registered new tserver with Master: 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
I20260809 19:24:01.612257 1283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:60949
I20260809 19:24:01.613955 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.607467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55290 (local address 127.9.91.126:43455)
0809 19:24:01.607603 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.607605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.607691 (+ 86us) server_negotiation.cc:407] Connection header received
0809 19:24:01.607753 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.607754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.607765 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.607814 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:01.608597 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.609061 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.610009 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.610289 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.610352 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.611658 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:01.611659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:01.611659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:01.611667 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:01.613075 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.613399 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.613400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.613401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.613459 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.613602 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.613603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.613603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.613692 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:01.613802 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.613830 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.613894 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260809 19:24:01.614631 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:24:01.616984 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.614956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.9.91.126:43455)
0809 19:24:01.615004 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.615006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.615047 (+ 41us) server_negotiation.cc:407] Connection header received
0809 19:24:01.615113 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.615114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.615140 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.615191 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:01.615288 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.615713 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.616367 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.616549 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.616602 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.616710 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:01.616745 (+ 35us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 19:24:01.616943 (+ 198us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55302: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:24:01.617041 2024 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55302: token expired
W20260809 19:24:01.617055 1182 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.91.126:43455)
I20260809 19:24:01.622476 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.617278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.9.91.126:43455)
0809 19:24:01.617313 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.617315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.617325 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:24:01.617398 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.617399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.617408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.617463 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:01.617927 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.618368 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.619117 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.619310 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.619370 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.620381 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:01.620382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:01.620382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:01.620389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:01.621408 (+ 1019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.621745 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.621746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.621746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.621802 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:01.621999 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:01.622000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:01.622000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:01.622105 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:01.622190 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.622327 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.622397 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:24:01.623623 1283 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:24:01.625169 1283 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:24:01.627188 1477 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf (DEFAULT_TABLE table=test-table [id=1fcf27dfce3a41e8a34bddba3f35c72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:01.627395 1477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:01.628808 1415 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Bootstrap starting.
I20260809 19:24:01.629145 1415 tablet_bootstrap.cc:654] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:01.630080 1415 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: No bootstrap required, opened a new log
I20260809 19:24:01.630126 1415 ts_tablet_manager.cc:1397] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:01.630275 1415 raft_consensus.cc:348] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.630336 1415 raft_consensus.cc:374] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:01.630360 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.627041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40033 (local address 127.0.0.1:44102)
0809 19:24:01.627283 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.627292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.627355 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.627574 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.627575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.627578 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:01.627621 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.627623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.628591 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.628592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.629466 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.629469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.630096 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.630097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.630163 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.630182 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.630242 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.630287 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
I20260809 19:24:01.630374 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.627059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.9.91.67:40033)
0809 19:24:01.627230 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.627234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.627354 (+ 120us) server_negotiation.cc:407] Connection header received
0809 19:24:01.627455 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.627458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.627474 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.627554 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:01.627680 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.628491 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.629546 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.630004 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.630077 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.630133 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.630267 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.630303 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
I20260809 19:24:01.630380 1415 raft_consensus.cc:729] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae8df6469134ce1916421235be68f2b, State: Initialized, Role: FOLLOWER
I20260809 19:24:01.630429 1415 consensus_queue.cc:260] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.630513 1415 ts_tablet_manager.cc:1428] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:01.631311 1904 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf (DEFAULT_TABLE table=test-table [id=1fcf27dfce3a41e8a34bddba3f35c72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:01.631623 1904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:01.632458 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.626855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:33473 (local address 127.0.0.1:50396)
0809 19:24:01.626885 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.626894 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.628838 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.629151 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.629152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.629154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:01.629191 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.629193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.630380 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.630381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.631344 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.631347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.632204 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.632205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.632277 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.632282 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.632348 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.632394 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 19:24:01.632465 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.627013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50396 (local address 127.9.91.66:33473)
0809 19:24:01.628988 (+ 1975us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.628993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.629002 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:01.629040 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.629042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.629057 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.629137 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:01.629258 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.630273 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.631498 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.632112 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.632169 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.632206 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.632368 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.632396 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1930,"thread_start_us":111,"threads_started":1}
I20260809 19:24:01.633191 1694 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf (DEFAULT_TABLE table=test-table [id=1fcf27dfce3a41e8a34bddba3f35c72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:01.633538 1694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:01.634954 2034 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Bootstrap starting.
I20260809 19:24:01.635639 2034 tablet_bootstrap.cc:654] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:01.636085 2034 log.cc:824] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:01.636711 2034 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: No bootstrap required, opened a new log
I20260809 19:24:01.636785 2034 ts_tablet_manager.cc:1397] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:01.636823 2037 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Bootstrap starting.
I20260809 19:24:01.637456 2037 tablet_bootstrap.cc:654] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:01.638072 2037 log.cc:824] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:01.638592 2034 raft_consensus.cc:348] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.638761 2034 raft_consensus.cc:374] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:01.638809 2034 raft_consensus.cc:729] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd2119e0b9d412cad61fe3df5d82b7c, State: Initialized, Role: FOLLOWER
I20260809 19:24:01.638839 2037 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: No bootstrap required, opened a new log
I20260809 19:24:01.638891 2037 ts_tablet_manager.cc:1397] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:01.638950 2034 consensus_queue.cc:260] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.639250 2015 heartbeater.cc:503] Master 127.9.91.126:43455 was elected leader, sending a full tablet report...
I20260809 19:24:01.639248 2034 ts_tablet_manager.cc:1428] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:24:01.640630 2037 raft_consensus.cc:348] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.640765 2037 raft_consensus.cc:374] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:01.640787 2037 raft_consensus.cc:729] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9c085b6ba53406abf487163fcb692dc, State: Initialized, Role: FOLLOWER
I20260809 19:24:01.640858 2037 consensus_queue.cc:260] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.641041 2037 ts_tablet_manager.cc:1428] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:01.641121 1802 heartbeater.cc:503] Master 127.9.91.126:43455 was elected leader, sending a full tablet report...
I20260809 19:24:01.661664 1593 raft_consensus.cc:482] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:01.661775 1593 raft_consensus.cc:504] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.662076 1593 leader_election.cc:288] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
I20260809 19:24:01.665675 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.662413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:50859 (local address 127.9.91.67:40033)
0809 19:24:01.662463 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.662464 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.662568 (+ 104us) server_negotiation.cc:407] Connection header received
0809 19:24:01.662616 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.662616 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.662625 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.662661 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:01.662804 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.663489 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.664747 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.665310 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.665381 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.665389 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.665581 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.665614 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:24:01.665683 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.662400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:55389 (local address 127.9.91.66:33473)
0809 19:24:01.662439 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.662440 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.662448 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:01.662468 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.662469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.662477 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.662523 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:01.662710 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.663399 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.664749 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.665310 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.665381 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.665389 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.665581 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.665614 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:24:01.665704 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.662212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:33473 (local address 127.9.91.65:55389)
0809 19:24:01.662261 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.662272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.662319 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.662560 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.662561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.662563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:01.662631 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.662633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.663476 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.663476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.664655 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.664658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.665393 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.665395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.665474 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.665498 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.665565 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.665610 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":3}
I20260809 19:24:01.665730 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.662316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40033 (local address 127.9.91.65:50859)
0809 19:24:01.662475 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.662485 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:01.662560 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:01.662672 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:01.662673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:01.662675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:01.662735 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.662736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.663557 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.663558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.664656 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:01.664658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.665416 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.665417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:01.665474 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.665505 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 19:24:01.665565 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:24:01.665610 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260809 19:24:01.666042 1729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3d4b3c7d0a4167af4a93a47b5e85cf" candidate_uuid: "0ae8df6469134ce1916421235be68f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c085b6ba53406abf487163fcb692dc" is_pre_election: true
I20260809 19:24:01.666150 1729 raft_consensus.cc:2454] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae8df6469134ce1916421235be68f2b in term 0.
I20260809 19:24:01.666131 1942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3d4b3c7d0a4167af4a93a47b5e85cf" candidate_uuid: "0ae8df6469134ce1916421235be68f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" is_pre_election: true
I20260809 19:24:01.666247 1942 raft_consensus.cc:2454] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae8df6469134ce1916421235be68f2b in term 0.
I20260809 19:24:01.666368 1411 leader_election.cc:302] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b, b9c085b6ba53406abf487163fcb692dc; no voters:
I20260809 19:24:01.666491 1593 raft_consensus.cc:2781] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:24:01.666553 1593 raft_consensus.cc:482] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:24:01.666577 1593 raft_consensus.cc:3037] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:01.667212 1593 raft_consensus.cc:504] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.667327 1593 leader_election.cc:288] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 1 election: Requested vote from peers b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
I20260809 19:24:01.667495 1729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3d4b3c7d0a4167af4a93a47b5e85cf" candidate_uuid: "0ae8df6469134ce1916421235be68f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9c085b6ba53406abf487163fcb692dc"
I20260809 19:24:01.667538 1942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3d4b3c7d0a4167af4a93a47b5e85cf" candidate_uuid: "0ae8df6469134ce1916421235be68f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c"
I20260809 19:24:01.667570 1729 raft_consensus.cc:3037] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:01.667594 1942 raft_consensus.cc:3037] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:01.668322 1729 raft_consensus.cc:2454] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae8df6469134ce1916421235be68f2b in term 1.
I20260809 19:24:01.668322 1942 raft_consensus.cc:2454] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae8df6469134ce1916421235be68f2b in term 1.
I20260809 19:24:01.668483 1411 leader_election.cc:302] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b, b9c085b6ba53406abf487163fcb692dc; no voters:
I20260809 19:24:01.668584 1593 raft_consensus.cc:2781] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:01.668711 1593 raft_consensus.cc:686] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 LEADER]: Becoming Leader. State: Replica: 0ae8df6469134ce1916421235be68f2b, State: Running, Role: LEADER
I20260809 19:24:01.668761 1593 consensus_queue.cc:237] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:01.669485 1283 catalog_manager.cc:5495] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae8df6469134ce1916421235be68f2b (127.9.91.65). New cstate: current_term: 1 leader_uuid: "0ae8df6469134ce1916421235be68f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } health_report { overall_health: HEALTHY } } }
I20260809 19:24:01.681372 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.678957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.9.91.65:35821)
0809 19:24:01.679002 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:01.679004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:01.679014 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:24:01.679093 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:01.679094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:01.679112 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:01.679178 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:01.679303 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.679859 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.680584 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:01.680837 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:01.680900 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:01.681071 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:01.681161 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:24:01.681227 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:01.681289 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 19:24:01.681309 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:24:01.682583 1942 raft_consensus.cc:1261] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Refusing update from remote peer 0ae8df6469134ce1916421235be68f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:01.682583 1729 raft_consensus.cc:1261] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 1 FOLLOWER]: Refusing update from remote peer 0ae8df6469134ce1916421235be68f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:01.682898 2042 consensus_queue.cc:1035] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:01.683061 1593 consensus_queue.cc:1035] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:01.687702 1281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:01.687773 1281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:01.688402 1281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Sending DeleteTablet for 3 replicas of tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:01.688665 1477 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:01 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50770
I20260809 19:24:01.688764 1596 tablet_replica.cc:330] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: stopping tablet replica
I20260809 19:24:01.688807 1596 raft_consensus.cc:2229] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:24:01.688879 1596 raft_consensus.cc:2258] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:01.688851 1694 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:01 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:50396
I20260809 19:24:01.689091 1904 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:01 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:44102
I20260809 19:24:01.689138 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 1347
I20260809 19:24:01.689306 2063 tablet_replica.cc:330] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: stopping tablet replica
I20260809 19:24:01.689390 2063 raft_consensus.cc:2229] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:01.689479 2063 raft_consensus.cc:2258] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:01.691550 2064 tablet_replica.cc:330] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: stopping tablet replica
I20260809 19:24:01.691623 2064 raft_consensus.cc:2229] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:01.691689 2064 raft_consensus.cc:2258] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:01.693915 2064 ts_tablet_manager.cc:1905] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:01.695233 2064 ts_tablet_manager.cc:1918] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:01.695289 2064 log.cc:1196] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:01.695494 2064 ts_tablet_manager.cc:1939] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 6cd2119e0b9d412cad61fe3df5d82b7c: Deleting consensus metadata
I20260809 19:24:01.695845 1264 catalog_manager.cc:4841] TS 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033): tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf (table test-table [id=1fcf27dfce3a41e8a34bddba3f35c72e]) successfully deleted
I20260809 19:24:01.696863 2063 ts_tablet_manager.cc:1905] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:01.698073 2063 ts_tablet_manager.cc:1918] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:01.698144 2063 log.cc:1196] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:01.698406 2063 ts_tablet_manager.cc:1939] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P b9c085b6ba53406abf487163fcb692dc: Deleting consensus metadata
I20260809 19:24:01.698771 1264 catalog_manager.cc:4841] TS b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473): tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf (table test-table [id=1fcf27dfce3a41e8a34bddba3f35c72e]) successfully deleted
W20260809 19:24:01.705660 1261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:35821 (error 108)
I20260809 19:24:01.706049 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 1592
I20260809 19:24:01.706146 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:01.706000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:35821 (local address 127.0.0.1:50800)
0809 19:24:01.706065 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:01.706114 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:35821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 19:24:01.709677 1261 catalog_manager.cc:4584] TS 0ae8df6469134ce1916421235be68f2b (127.9.91.65:35821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a3d4b3c7d0a4167af4a93a47b5e85cf: Network error: Client connection negotiation failed: client connection to 127.9.91.65:35821: connect: Connection refused (error 111)
I20260809 19:24:01.713985 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 1808
I20260809 19:24:01.723675 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 1237
I20260809 19:24:21.732815 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43455
--webserver_interface=127.9.91.126
--webserver_port=45229
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:43455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:21.813357 2066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:21.813537 2066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:21.813558 2066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:21.814646 2066 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 19:24:21.814683 2066 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 19:24:21.814704 2066 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 19:24:21.814726 2066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:21.814749 2066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:21.814771 2066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:21.814786 2066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:21.814810 2066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:21.816294 2066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:43455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=45229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:21.816509 2066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:21.816711 2066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 19:24:21.821444 2066 server_base.cc:1034] running on GCE node
W20260809 19:24:21.821460 2072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:21.821511 2076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:21.821736 2074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:21.821965 2066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:21.822228 2066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:21.823357 2066 hybrid_clock.cc:648] HybridClock initialized: now 1786303461823346 us; error 30 us; skew 500 ppm
Aug 09 19:24:21 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303461, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:21.828172 2066 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:24:21.828598 2066 webserver.cc:460] Webserver started at http://127.9.91.126:45229/ using document root <none> and password file <none>
I20260809 19:24:21.828809 2066 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:21.828853 2066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:21.830101 2066 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:21.830698 2083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:21.830832 2066 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:21.830904 2066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "88945e2ec8754344b1b9269584d3f187"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "1e9237108d350278914388ba1f446876"
server_key_iv: "d838ee976c908cfb21d97a14c3c007f4"
server_key_version: "encryptionkey@0"
I20260809 19:24:21.831233 2066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:21.839779 2066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:21.841279 2066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:21.841403 2066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:21.845152 2066 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:43455
I20260809 19:24:21.845186 2169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:43455 every 8 connection(s)
I20260809 19:24:21.845507 2066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:21.846784 2171 sys_catalog.cc:263] Verifying existing consensus state
I20260809 19:24:21.847456 2171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Bootstrap starting.
I20260809 19:24:21.848374 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 2066
I20260809 19:24:21.848611 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:35821
--local_ip_for_outbound_sockets=127.9.91.65
--tserver_master_addrs=127.9.91.126:43455
--webserver_port=34355
--webserver_interface=127.9.91.65
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:24:21.856907 2171 log.cc:824] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:21.859091 2171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:24:21.859355 2171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Bootstrap complete.
I20260809 19:24:21.861379 2171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:21.861584 2171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88945e2ec8754344b1b9269584d3f187, State: Initialized, Role: FOLLOWER
I20260809 19:24:21.861728 2171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:21.861788 2171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:21.861827 2171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:21.861872 2171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 2 FOLLOWER]: Advancing to term 3
I20260809 19:24:21.862795 2171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:21.862883 2171 leader_election.cc:302] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88945e2ec8754344b1b9269584d3f187; no voters:
I20260809 19:24:21.863044 2171 leader_election.cc:288] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 19:24:21.863097 2178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 19:24:21.863219 2178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [term 3 LEADER]: Becoming Leader. State: Replica: 88945e2ec8754344b1b9269584d3f187, State: Running, Role: LEADER
I20260809 19:24:21.863306 2171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:21.863365 2178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } }
I20260809 19:24:21.863679 2181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "88945e2ec8754344b1b9269584d3f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } } }
I20260809 19:24:21.863775 2181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:21.863948 2190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:21.863698 2178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88945e2ec8754344b1b9269584d3f187. Latest consensus state: current_term: 3 leader_uuid: "88945e2ec8754344b1b9269584d3f187" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88945e2ec8754344b1b9269584d3f187" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43455 } } }
I20260809 19:24:21.864130 2178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:21.864959 2190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:21.865226 2190 catalog_manager.cc:1259] Loaded cluster ID: b3ecaf43a48c4536b1eaad3fa9521bde
I20260809 19:24:21.865269 2190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:21.866030 2190 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:21.866189 2190 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Loaded TSK: 0
I20260809 19:24:21.866262 2190 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88945e2ec8754344b1b9269584d3f187: Loaded TSK: 1
F20260809 19:24:21.866477 2190 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6017c439ad google::LogMessage::Fail() at ??:0
@ 0x7f6017c4688a google::LogMessage::SendToLog() at ??:0
@ 0x7f6017c434de google::LogMessage::Flush() at ??:0
@ 0x7f6017c44a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f601a3b0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f601a3add41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f601fb9d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f601fb9d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f601fb9d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f601fb63c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f601fb6dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6019b110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6019b06954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f601a96a6db start_thread at ??:0
@ 0x7f601674161f clone at ??:0
W20260809 19:24:21.942428 2174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:21.942612 2174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:21.942642 2174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:21.943771 2174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:21.943813 2174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:21.943847 2174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:21.945250 2174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:35821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=34355
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:21.945462 2174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:21.945654 2174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:21.950246 2246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:21.950273 2248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:21.950289 2245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:21.950587 2174 server_base.cc:1034] running on GCE node
I20260809 19:24:21.950809 2174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:21.951072 2174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:21.952244 2174 hybrid_clock.cc:648] HybridClock initialized: now 1786303461952226 us; error 34 us; skew 500 ppm
Aug 09 19:24:21 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303461, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:21.955302 2174 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:24:21.955806 2174 webserver.cc:460] Webserver started at http://127.9.91.65:34355/ using document root <none> and password file <none>
I20260809 19:24:21.956037 2174 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:21.956096 2174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:21.957760 2174 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:24:21.958489 2259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:21.958659 2174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:21.958745 2174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "0ae8df6469134ce1916421235be68f2b"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "466c5624aa736979023e7913b0c574f1"
server_key_iv: "0f2a5c086ee56dd3b1bbb6aadb80e8ea"
server_key_version: "encryptionkey@0"
I20260809 19:24:21.959084 2174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:21.970366 2174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:21.972368 2174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:21.972482 2174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:21.972681 2174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:21.973078 2267 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 19:24:21.974895 2174 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 19:24:21.974963 2174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 19:24:21.974995 2174 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 19:24:21.975598 2174 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 19:24:21.975630 2174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 19:24:21.975742 2267 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Bootstrap starting.
I20260809 19:24:21.982295 2174 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:35821
I20260809 19:24:21.982642 2174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:24:21.983978 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 2174
I20260809 19:24:21.984234 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:33473
--local_ip_for_outbound_sockets=127.9.91.66
--tserver_master_addrs=127.9.91.126:43455
--webserver_port=34039
--webserver_interface=127.9.91.66
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:24:21.986853 2267 log.cc:824] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:21.986951 2438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:35821 every 8 connection(s)
I20260809 19:24:21.989418 2267 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 19:24:21.989741 2267 tablet_bootstrap.cc:492] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Bootstrap complete.
I20260809 19:24:21.990964 2267 ts_tablet_manager.cc:1397] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.013s
I20260809 19:24:21.992336 2267 raft_consensus.cc:348] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:21.992655 2267 raft_consensus.cc:729] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae8df6469134ce1916421235be68f2b, State: Initialized, Role: FOLLOWER
I20260809 19:24:21.992825 2267 consensus_queue.cc:260] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:21.993083 2267 ts_tablet_manager.cc:1428] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260809 19:24:22.059757 2441 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:24:21.983259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:52455)
0809 19:24:21.983426 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:21.983466 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:21.983914 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:22.059540 (+ 75626us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
W20260809 19:24:22.060025 2439 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:43455 (0 consecutive failures): Network error: Failed to ping master at 127.9.91.126:43455: Client connection negotiation failed: client connection to 127.9.91.126:43455: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 19:24:22.060390 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.060260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:43929)
0809 19:24:22.060317 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.060363 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 19:24:22.060738 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.060653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:35597)
0809 19:24:22.060690 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.060718 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:24:22.060855 2439 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 19:24:22.067466 2442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:22.067631 2442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:22.067651 2442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:22.068735 2442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:22.068768 2442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:22.068792 2442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:22.070129 2442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:33473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=34039
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:22.070298 2442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:22.070462 2442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:22.074819 2452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:22.074849 2457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:22.074887 2451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:22.075034 2442 server_base.cc:1034] running on GCE node
I20260809 19:24:22.075217 2442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:22.075435 2442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:22.076570 2442 hybrid_clock.cc:648] HybridClock initialized: now 1786303462076552 us; error 32 us; skew 500 ppm
Aug 09 19:24:22 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303462, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:22.078791 2442 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:24:22.079293 2442 webserver.cc:460] Webserver started at http://127.9.91.66:34039/ using document root <none> and password file <none>
I20260809 19:24:22.079514 2442 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:22.079566 2442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:22.080775 2442 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:22.081265 2471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:22.081409 2442 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:22.081486 2442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "b9c085b6ba53406abf487163fcb692dc"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "ff15284edc24b39345abb4eedf827138"
server_key_iv: "0ccfee673ab4d76797f668e6f9674856"
server_key_version: "encryptionkey@0"
I20260809 19:24:22.081780 2442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:22.096225 2442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:22.097734 2442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:22.097856 2442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:22.098040 2442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:22.098294 2442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:22.098326 2442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:22.098351 2442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:22.098368 2442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:22.103747 2442 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:33473
I20260809 19:24:22.103806 2659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:33473 every 8 connection(s)
I20260809 19:24:22.104081 2442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:24:22.104950 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.104609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:52373)
0809 19:24:22.104751 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.104817 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
W20260809 19:24:22.105259 2660 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:43455 (0 consecutive failures): Network error: Failed to ping master at 127.9.91.126:43455: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
I20260809 19:24:22.105471 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.105408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:52551)
0809 19:24:22.105433 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.105456 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260809 19:24:22.105654 2663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.105609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:52141)
0809 19:24:22.105627 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.105642 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 19:24:22.105738 2660 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:24:22.109887 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 2442
I20260809 19:24:22.110138 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:40033
--local_ip_for_outbound_sockets=127.9.91.67
--tserver_master_addrs=127.9.91.126:43455
--webserver_port=41013
--webserver_interface=127.9.91.67
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:22.190946 2664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:22.191144 2664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:22.191166 2664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:22.192257 2664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:22.192291 2664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:22.192315 2664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:22.193701 2664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:40033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=41013
--tserver_master_addrs=127.9.91.126:43455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:22.193881 2664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:22.194048 2664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:22.198735 2678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:22.198743 2673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:22.198742 2676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:22.198894 2664 server_base.cc:1034] running on GCE node
I20260809 19:24:22.199120 2664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:22.199357 2664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:22.200503 2664 hybrid_clock.cc:648] HybridClock initialized: now 1786303462200490 us; error 26 us; skew 500 ppm
Aug 09 19:24:22 dist-test-slave-1vmg krb5kdc[381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303462, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:22.202884 2664 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:24:22.203306 2664 webserver.cc:460] Webserver started at http://127.9.91.67:41013/ using document root <none> and password file <none>
I20260809 19:24:22.203505 2664 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:22.203552 2664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:22.204742 2664 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:22.205304 2690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:22.205446 2664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:22.205528 2664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "6cd2119e0b9d412cad61fe3df5d82b7c"
format_stamp: "Formatted at 2026-08-09 19:23:40 on dist-test-slave-1vmg"
server_key: "504f28d9ed415bcb466838ae66396d8c"
server_key_iv: "0b0a432f4f1f9c68ba3fa240b8202dfb"
server_key_version: "encryptionkey@0"
I20260809 19:24:22.205821 2664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:22.216481 2664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:22.218454 2664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:22.218583 2664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:22.218771 2664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:22.219077 2664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:22.219120 2664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:22.219146 2664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:22.219169 2664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:22.224895 2664 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:40033
I20260809 19:24:22.224929 2880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:40033 every 8 connection(s)
I20260809 19:24:22.225246 2664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:24:22.226152 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.225819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:36577)
0809 19:24:22.225958 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.226021 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
W20260809 19:24:22.226503 2885 heartbeater.cc:644] Failed to heartbeat to 127.9.91.126:43455 (0 consecutive failures): Network error: Failed to ping master at 127.9.91.126:43455: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
I20260809 19:24:22.226729 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.226652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:47855)
0809 19:24:22.226688 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.226712 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 19:24:22.226969 2890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:22.226874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:38511)
0809 19:24:22.226902 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:22.226950 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 19:24:22.227058 2885 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 19:24:22.235301 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 2664
I20260809 19:24:23.061375 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:23.061131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:37619)
0809 19:24:23.061280 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:23.061314 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260809 19:24:23.106195 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:23.106013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:53833)
0809 19:24:23.106116 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:23.106145 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
I20260809 19:24:23.227519 2901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:23.227325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:54523)
0809 19:24:23.227436 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:23.227469 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260809 19:24:23.527864 2903 raft_consensus.cc:482] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:23.528008 2903 raft_consensus.cc:504] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:23.528312 2903 leader_election.cc:288] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
Aug 09 19:24:23 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303461, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.66@KRBTEST.COM
Aug 09 19:24:23 dist-test-slave-1vmg krb5kdc[381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303461, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.67@KRBTEST.COM
I20260809 19:24:23.538295 2904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:23.528564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40033 (local address 127.9.91.65:35955)
0809 19:24:23.528847 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:23.528858 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:23.528921 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:23.529568 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:23.529569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:23.529876 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:23.530064 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:23.530071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.531180 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.531180 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:23.531998 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:23.532001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.532561 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.532562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:23.532669 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:23.533244 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:23.533249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:23.535031 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:23.536968 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:23.536969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:23.536970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:23.537127 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:23.537353 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:23.537354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:23.537354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:23.537414 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:23.537832 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:23.537833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:23.537904 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 19:24:23.538007 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 19:24:23.538198 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":47,"thread_start_us":68,"threads_started":1}
I20260809 19:24:23.538311 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:23.528704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35955 (local address 127.9.91.67:40033)
0809 19:24:23.528833 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:23.528845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:23.528906 (+ 61us) server_negotiation.cc:407] Connection header received
0809 19:24:23.529409 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:23.529411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:23.529451 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:23.529574 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:23.530201 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.531082 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.532080 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.532479 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.532580 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:23.535129 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:23.535133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:23.535133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:23.535145 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:23.536900 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:23.537204 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:23.537207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:23.537207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:23.537295 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:23.537467 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:23.537468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:23.537468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:23.537727 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:23.537801 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:23.538075 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 19:24:23.538193 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260809 19:24:23.538329 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:23.528599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:42539 (local address 127.9.91.66:33473)
0809 19:24:23.528770 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:23.528782 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:23.528814 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:24:23.529269 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:23.529271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:23.529314 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:23.529451 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:23.530185 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.531087 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.532105 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.532488 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.532593 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:23.534797 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:23.534801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:23.534802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:23.534815 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:23.536651 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:23.537062 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:23.537066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:23.537066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:23.537169 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:23.537405 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:23.537406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:23.537406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:23.537669 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:23.537752 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:23.538084 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 19:24:23.538215 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
I20260809 19:24:23.538470 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:23.528454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:33473 (local address 127.9.91.65:42539)
0809 19:24:23.528519 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:23.528534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:23.528610 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:23.529508 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:23.529511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:23.529875 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:23.530064 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:23.530071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.531191 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.531192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:23.532017 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:23.532019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:23.532584 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:23.532585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:23.532670 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:23.533190 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:23.533198 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:23.534684 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:23.536741 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:23.536748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:23.536750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:23.536968 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:23.537236 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:23.537237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:23.537237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:23.537342 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:23.537746 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:23.537753 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:23.537902 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 19:24:23.538008 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 19:24:23.538279 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":181}
W20260809 19:24:23.539600 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
W20260809 19:24:23.539682 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:23.539726 2263 leader_election.cc:302] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b; no voters: 6cd2119e0b9d412cad61fe3df5d82b7c, b9c085b6ba53406abf487163fcb692dc
I20260809 19:24:23.540216 2903 raft_consensus.cc:2726] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:24:24.061944 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:24.061753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:36507)
0809 19:24:24.061863 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:24.061895 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260809 19:24:24.106724 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:24.106537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:44725)
0809 19:24:24.106654 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:24.106687 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260809 19:24:24.228099 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:24.227909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:55633)
0809 19:24:24.228029 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:24.228061 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260809 19:24:25.062537 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:25.062313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:53977)
0809 19:24:25.062450 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:25.062485 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260809 19:24:25.107306 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:25.107103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:34519)
0809 19:24:25.107234 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:25.107266 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260809 19:24:25.228711 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:25.228512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:52681)
0809 19:24:25.228636 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:25.228671 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260809 19:24:25.394430 2920 raft_consensus.cc:482] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:25.394523 2920 raft_consensus.cc:504] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:25.394678 2920 leader_election.cc:288] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
W20260809 19:24:25.395093 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
W20260809 19:24:25.395180 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:25.395205 2263 leader_election.cc:302] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b; no voters: 6cd2119e0b9d412cad61fe3df5d82b7c, b9c085b6ba53406abf487163fcb692dc
I20260809 19:24:25.395288 2920 raft_consensus.cc:2726] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:24:26.063151 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:26.062935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:50267)
0809 19:24:26.063059 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:26.063098 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260809 19:24:26.107808 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:26.107641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:45197)
0809 19:24:26.107745 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:26.107773 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260809 19:24:26.229313 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:26.229107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:59077)
0809 19:24:26.229241 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:26.229274 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260809 19:24:27.063793 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:27.063590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:54821)
0809 19:24:27.063709 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:27.063743 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260809 19:24:27.108355 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:27.108158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:49497)
0809 19:24:27.108275 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:27.108314 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260809 19:24:27.229939 2930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:27.229695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:58783)
0809 19:24:27.229864 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:27.229897 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260809 19:24:27.771667 2931 raft_consensus.cc:482] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:27.771759 2931 raft_consensus.cc:504] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:27.771893 2931 leader_election.cc:288] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
W20260809 19:24:27.772229 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
W20260809 19:24:27.772305 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:27.772325 2263 leader_election.cc:302] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b; no voters: 6cd2119e0b9d412cad61fe3df5d82b7c, b9c085b6ba53406abf487163fcb692dc
I20260809 19:24:27.772390 2931 raft_consensus.cc:2726] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:24:28.064422 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:28.064214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:54655)
0809 19:24:28.064337 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:28.064373 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260809 19:24:28.108875 2933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:28.108697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:48763)
0809 19:24:28.108804 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:28.108836 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260809 19:24:28.230497 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:28.230302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:42181)
0809 19:24:28.230423 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:28.230454 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260809 19:24:29.065068 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:29.064863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:42091)
0809 19:24:29.064982 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:29.065018 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260809 19:24:29.109426 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:29.109236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:47149)
0809 19:24:29.109353 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:29.109384 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260809 19:24:29.231060 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:29.230856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:54493)
0809 19:24:29.230986 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:29.231019 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260809 19:24:30.065747 2940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:30.065488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:56257)
0809 19:24:30.065645 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:30.065696 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":71,"threads_started":1}
I20260809 19:24:30.109965 2941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:30.109789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:43691)
0809 19:24:30.109899 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:30.109928 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260809 19:24:30.231666 2944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:30.231490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:35079)
0809 19:24:30.231595 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:30.231627 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":59,"threads_started":1}
I20260809 19:24:31.066332 2946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:31.066123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:48039)
0809 19:24:31.066245 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:31.066281 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260809 19:24:31.110491 2950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:31.110318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:42743)
0809 19:24:31.110420 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:31.110453 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260809 19:24:31.161019 2951 raft_consensus.cc:482] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:31.161098 2951 raft_consensus.cc:504] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:31.161266 2951 leader_election.cc:288] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
W20260809 19:24:31.161607 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
W20260809 19:24:31.161697 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:31.161729 2263 leader_election.cc:302] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b; no voters: 6cd2119e0b9d412cad61fe3df5d82b7c, b9c085b6ba53406abf487163fcb692dc
I20260809 19:24:31.161821 2951 raft_consensus.cc:2726] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:24:31.232257 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:31.232047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:57147)
0809 19:24:31.232165 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:31.232207 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260809 19:24:32.067196 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:32.066953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:44243)
0809 19:24:32.067101 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:32.067146 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260809 19:24:32.111100 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:32.110869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:51019)
0809 19:24:32.111020 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:32.111061 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260809 19:24:32.232885 2963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:32.232667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:54617)
0809 19:24:32.232807 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:32.232846 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260809 19:24:33.067956 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:33.067728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:55845)
0809 19:24:33.067879 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:33.067909 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260809 19:24:33.111650 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:33.111471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:39951)
0809 19:24:33.111579 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:33.111611 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260809 19:24:33.233448 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:33.233261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:57425)
0809 19:24:33.233378 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:33.233408 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260809 19:24:34.068516 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:34.068319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:39481)
0809 19:24:34.068434 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:34.068466 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260809 19:24:34.112190 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:34.112009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:40441)
0809 19:24:34.112112 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:34.112153 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
I20260809 19:24:34.233978 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:34.233793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:57379)
0809 19:24:34.233902 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:34.233937 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260809 19:24:35.069069 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:35.068870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:43673)
0809 19:24:35.068989 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:35.069023 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260809 19:24:35.112685 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:35.112514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:52131)
0809 19:24:35.112617 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:35.112648 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260809 19:24:35.234562 2989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:35.234376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:34511)
0809 19:24:35.234486 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:35.234520 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260809 19:24:36.069674 2990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:36.069464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:37041)
0809 19:24:36.069591 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:36.069625 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260809 19:24:36.113322 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:36.113122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:59625)
0809 19:24:36.113252 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:36.113284 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260809 19:24:36.235131 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:36.234941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:47935)
0809 19:24:36.235056 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:36.235090 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260809 19:24:37.070338 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:37.070131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.65:51407)
0809 19:24:37.070262 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:37.070291 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260809 19:24:37.113945 3000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:37.113753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.66:42573)
0809 19:24:37.113875 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:37.113906 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260809 19:24:37.162710 3001 raft_consensus.cc:482] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:37.162793 3001 raft_consensus.cc:504] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9c085b6ba53406abf487163fcb692dc" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33473 } } peers { permanent_uuid: "6cd2119e0b9d412cad61fe3df5d82b7c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40033 } } peers { permanent_uuid: "0ae8df6469134ce1916421235be68f2b" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 35821 } }
I20260809 19:24:37.162969 3001 leader_election.cc:288] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473), 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033)
W20260809 19:24:37.163340 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9c085b6ba53406abf487163fcb692dc (127.9.91.66:33473): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
W20260809 19:24:37.163414 2263 leader_election.cc:341] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6cd2119e0b9d412cad61fe3df5d82b7c (127.9.91.67:40033): Not found: Tablet not found: 0a3d4b3c7d0a4167af4a93a47b5e85cf
I20260809 19:24:37.163430 2263 leader_election.cc:302] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ae8df6469134ce1916421235be68f2b; no voters: 6cd2119e0b9d412cad61fe3df5d82b7c, b9c085b6ba53406abf487163fcb692dc
I20260809 19:24:37.163498 3001 raft_consensus.cc:2726] T 0a3d4b3c7d0a4167af4a93a47b5e85cf P 0ae8df6469134ce1916421235be68f2b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 19:24:37.235762 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:37.235544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43455 (local address 127.9.91.67:60565)
0809 19:24:37.235680 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:37.235721 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.126:43455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260809 19:24:37.242656 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 2174
I20260809 19:24:37.253171 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 2442
I20260809 19:24:37.264317 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 2664
2026-08-09T19:24:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58198 ms)
[----------] 30 tests from SecurityITest (228654 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:37 dist-test-slave-1vmg krb5kdc[3005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:37 dist-test-slave-1vmg krb5kdc[3005](info): set up 2 sockets
Aug 09 19:24:37 dist-test-slave-1vmg krb5kdc[3005](info): commencing operation
krb5kdc: starting...
W20260809 19:24:38.413218 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:38 dist-test-slave-1vmg krb5kdc[3005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:24:38.523433 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36281
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:38.601048 3027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:38.601230 3027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:38.601250 3027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:38.602303 3027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:38.602335 3027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:38.602348 3027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:38.602360 3027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:38.602370 3027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:38.603765 3027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:38.603943 3027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:38.604096 3027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:38.608837 3034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:38.608904 3036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:38.608937 3027 server_base.cc:1034] running on GCE node
W20260809 19:24:38.608871 3033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:38.609318 3027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:38.609584 3027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:38.610718 3027 hybrid_clock.cc:648] HybridClock initialized: now 1786303478610707 us; error 30 us; skew 500 ppm
Aug 09 19:24:38 dist-test-slave-1vmg krb5kdc[3005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303478, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:38.616458 3027 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:24:38.616853 3027 webserver.cc:460] Webserver started at http://127.9.91.126:42063/ using document root <none> and password file <none>
I20260809 19:24:38.617064 3027 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:38.617111 3027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:38.617230 3027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:38.618212 3027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "b8ec862c455a4eae80ff6e61db260854"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "19fc4600e837f6de232dfadb24b30240"
server_key_iv: "0c474eaa803f077c41adc443b325ee8a"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.618525 3027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "b8ec862c455a4eae80ff6e61db260854"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "19fc4600e837f6de232dfadb24b30240"
server_key_iv: "0c474eaa803f077c41adc443b325ee8a"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.619769 3027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:24:38.620409 3052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:38.620553 3027 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:38.620635 3027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "b8ec862c455a4eae80ff6e61db260854"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "19fc4600e837f6de232dfadb24b30240"
server_key_iv: "0c474eaa803f077c41adc443b325ee8a"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.620689 3027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:38.644106 3027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:38.645624 3027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:38.645752 3027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:38.649415 3027 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:36281
I20260809 19:24:38.649448 3140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:36281 every 8 connection(s)
I20260809 19:24:38.649729 3027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:38.650478 3141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:38.652655 3141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854: Bootstrap starting.
I20260809 19:24:38.653183 3141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:38.653432 3141 log.cc:824] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:38.654049 3141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854: No bootstrap required, opened a new log
I20260809 19:24:38.655638 3141 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ec862c455a4eae80ff6e61db260854" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36281 } }
I20260809 19:24:38.655763 3141 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:38.655794 3141 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8ec862c455a4eae80ff6e61db260854, State: Initialized, Role: FOLLOWER
I20260809 19:24:38.655920 3141 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ec862c455a4eae80ff6e61db260854" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36281 } }
I20260809 19:24:38.655977 3141 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:38.656000 3141 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:38.656044 3141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:38.656507 3141 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ec862c455a4eae80ff6e61db260854" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36281 } }
I20260809 19:24:38.656601 3141 leader_election.cc:302] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8ec862c455a4eae80ff6e61db260854; no voters:
I20260809 19:24:38.656744 3141 leader_election.cc:288] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:38.656771 3146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:38.656909 3146 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [term 1 LEADER]: Becoming Leader. State: Replica: b8ec862c455a4eae80ff6e61db260854, State: Running, Role: LEADER
I20260809 19:24:38.656960 3141 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:38.656996 3146 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ec862c455a4eae80ff6e61db260854" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36281 } }
I20260809 19:24:38.657507 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 3027
I20260809 19:24:38.657630 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:38.657845 9581 external_mini_cluster.cc:1428] Setting key 33d66c2ac21ddcf40907d0f10e99286a
I20260809 19:24:38.658082 3147 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8ec862c455a4eae80ff6e61db260854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ec862c455a4eae80ff6e61db260854" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36281 } } }
I20260809 19:24:38.658454 3148 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8ec862c455a4eae80ff6e61db260854. Latest consensus state: current_term: 1 leader_uuid: "b8ec862c455a4eae80ff6e61db260854" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8ec862c455a4eae80ff6e61db260854" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36281 } } }
I20260809 19:24:38.658583 3147 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:38.658869 3148 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854 [sys.catalog]: This master's current role is: LEADER
W20260809 19:24:38.659097 3169 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:24:38.659191 3169 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:24:38.659299 3175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:38.659469 3175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:38.660933 3175 catalog_manager.cc:1347] Generated new cluster ID: 5b530c120afd4c1881332851c6ec7dbf
I20260809 19:24:38.660980 3175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:24:38 dist-test-slave-1vmg krb5kdc[3005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:38.668854 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:38.658452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.9.91.126:36281)
0809 19:24:38.658630 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:38.658640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:38.658822 (+ 182us) server_negotiation.cc:407] Connection header received
0809 19:24:38.658962 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:38.658969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:38.659028 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:38.659163 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:38.659834 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:38.660989 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:38.662058 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:38.662640 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:38.662732 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:38.665455 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:38.665460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:38.665460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:38.665473 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:38.667337 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:38.667688 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:38.667692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:38.667692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:38.667807 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:38.667995 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:38.667996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:38.667996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:38.668287 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:38.668437 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:38.668516 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:24:38.668657 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260809 19:24:38.669082 3175 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:38.669689 3175 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:38.673053 3175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854: Generated new TSK 0
I20260809 19:24:38.673189 3175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:24:38.703819 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36281
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:38.811327 3181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:38.811514 3181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:38.811545 3181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:38.812645 3181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:38.812685 3181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:38.812719 3181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:38.814165 3181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:38.814401 3181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:38.814642 3181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:38.819351 3193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:38.819466 3188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:38.819572 3189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:38.819883 3181 server_base.cc:1034] running on GCE node
I20260809 19:24:38.820073 3181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:38.820287 3181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:38.821425 3181 hybrid_clock.cc:648] HybridClock initialized: now 1786303478821407 us; error 38 us; skew 500 ppm
Aug 09 19:24:38 dist-test-slave-1vmg krb5kdc[3005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303478, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:38.829068 3181 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:24:38.829483 3181 webserver.cc:460] Webserver started at http://127.9.91.65:38517/ using document root <none> and password file <none>
I20260809 19:24:38.829700 3181 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:38.829749 3181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:38.829874 3181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:38.830837 3181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "a69852ba0b484674aaadffde2873f631"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "31daac44478d2dee9f95706c073c527a"
server_key_iv: "402455e53397fdbc75f77a4dfd899064"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.831205 3181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "a69852ba0b484674aaadffde2873f631"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "31daac44478d2dee9f95706c073c527a"
server_key_iv: "402455e53397fdbc75f77a4dfd899064"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.832489 3181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:38.833196 3211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:38.833352 3181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:38.833436 3181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "a69852ba0b484674aaadffde2873f631"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "31daac44478d2dee9f95706c073c527a"
server_key_iv: "402455e53397fdbc75f77a4dfd899064"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.833500 3181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:38.844071 3181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:38.846181 3181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:38.846294 3181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:38.846482 3181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:38.846763 3181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:38.846797 3181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:38.846822 3181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:38.846838 3181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:38.852408 3181 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:38315
I20260809 19:24:38.852450 3385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:38315 every 8 connection(s)
I20260809 19:24:38.852768 3181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:24:38 dist-test-slave-1vmg krb5kdc[3005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303478, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:38.858681 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 3181
I20260809 19:24:38.858800 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:24:38.859048 9581 external_mini_cluster.cc:1428] Setting key 1bf0866e6da707c4b5bf5a462d167850
I20260809 19:24:38.862267 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:38.853421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:54393 (local address 127.9.91.126:36281)
0809 19:24:38.853462 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:38.853464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:38.853667 (+ 203us) server_negotiation.cc:407] Connection header received
0809 19:24:38.854127 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:38.854128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:38.854143 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:38.854192 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:38.855087 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:38.855581 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:38.856539 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:38.856812 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:38.856863 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:38.859216 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:38.859218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:38.859218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:38.859226 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:38.860750 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:38.861191 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:38.861192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:38.861193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:38.861255 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:38.861577 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:38.861578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:38.861579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:38.861724 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:38.861801 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:38.862092 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 19:24:38.862179 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:24:38.862425 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:38.853409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36281 (local address 127.9.91.65:54393)
0809 19:24:38.853556 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:38.853597 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:38.854034 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:38.854256 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:38.854259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:38.854768 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:38.854987 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:38.854994 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:38.855663 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:38.855664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:38.856450 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:38.856453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:38.856895 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:38.856896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:38.857023 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:38.857455 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:38.857463 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:38.859062 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:38.860834 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:38.860842 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:38.860845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:38.861117 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:38.861334 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:38.861335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:38.861335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:38.861474 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:38.861798 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:38.861804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:38.861957 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 19:24:38.862063 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 19:24:38.862218 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260809 19:24:38.862967 3386 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36281
I20260809 19:24:38.863062 3386 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:38.863220 3386 heartbeater.cc:511] Master 127.9.91.126:36281 requested a full tablet report, sending...
I20260809 19:24:38.863710 3077 ts_manager.cc:194] Registered new tserver with Master: a69852ba0b484674aaadffde2873f631 (127.9.91.65:38315)
I20260809 19:24:38.864562 3077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:54393
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:24:38.885358 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36281
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:38.965406 3393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:38.965595 3393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:38.965626 3393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:38.966727 3393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:38.966765 3393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:38.966799 3393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:38.968207 3393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:38.968420 3393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:38.968622 3393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:38.973178 3402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:38.973232 3403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:38.973174 3405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:38.973225 3393 server_base.cc:1034] running on GCE node
I20260809 19:24:38.973533 3393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:38.973739 3393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:38.974870 3393 hybrid_clock.cc:648] HybridClock initialized: now 1786303478974858 us; error 31 us; skew 500 ppm
Aug 09 19:24:38 dist-test-slave-1vmg krb5kdc[3005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303478, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:38.977232 3393 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:24:38.977620 3393 webserver.cc:460] Webserver started at http://127.9.91.66:38805/ using document root <none> and password file <none>
I20260809 19:24:38.977784 3393 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:38.977820 3393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:38.977907 3393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:38.978790 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "a0bb9773d54d429ab9df067ab6d53913"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "d390e35566ca6e378cd46f9a21b1300a"
server_key_iv: "4150330715b2ac25dc5abe4584bb3ca6"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.979108 3393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "a0bb9773d54d429ab9df067ab6d53913"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "d390e35566ca6e378cd46f9a21b1300a"
server_key_iv: "4150330715b2ac25dc5abe4584bb3ca6"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.980301 3393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:38.980888 3417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:38.981034 3393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:38.981128 3393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "a0bb9773d54d429ab9df067ab6d53913"
format_stamp: "Formatted at 2026-08-09 19:24:38 on dist-test-slave-1vmg"
server_key: "d390e35566ca6e378cd46f9a21b1300a"
server_key_iv: "4150330715b2ac25dc5abe4584bb3ca6"
server_key_version: "encryptionkey@0"
I20260809 19:24:38.981184 3393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:38.990532 3393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:38.992452 3393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:38.992580 3393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:38.992756 3393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:38.993012 3393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:38.993052 3393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:38.993077 3393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:38.993103 3393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:38.998488 3393 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:37823
I20260809 19:24:38.998530 3606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:37823 every 8 connection(s)
I20260809 19:24:38.998773 3393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:24:38.999580 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 3393
I20260809 19:24:38.999703 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:24:38.999936 9581 external_mini_cluster.cc:1428] Setting key f9bac97f4ce0441da6fe45b00b9b1a20
Aug 09 19:24:39 dist-test-slave-1vmg krb5kdc[3005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303478, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:39.007860 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:38.999419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:33727 (local address 127.9.91.126:36281)
0809 19:24:38.999487 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:38.999490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:38.999687 (+ 197us) server_negotiation.cc:407] Connection header received
0809 19:24:39.000250 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.000251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.000263 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.000317 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.001379 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.001866 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.003034 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.003226 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.003281 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.005331 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.005333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.005333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:39.005341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.006621 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:39.006996 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:39.006998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:39.006998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:39.007058 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:39.007271 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:39.007272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:39.007272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:39.007374 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.007429 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.007681 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.007772 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:24:39.007987 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:38.999401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36281 (local address 127.9.91.66:33727)
0809 19:24:38.999559 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:38.999602 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.000170 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.000378 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.000382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.001044 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:39.001284 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.001291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.001944 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.001945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.002950 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.002953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.003294 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.003295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.003429 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.003827 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:39.003836 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:39.005244 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:39.006696 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:39.006703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:39.006705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:39.006905 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:39.007112 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:39.007112 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:39.007113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:39.007215 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:39.007429 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:39.007435 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:39.007567 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.007649 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.007793 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260809 19:24:39.008387 3608 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36281
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
I20260809 19:24:39.008493 3608 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:39.008673 3608 heartbeater.cc:511] Master 127.9.91.126:36281 requested a full tablet report, sending...
I20260809 19:24:39.009018 3077 ts_manager.cc:194] Registered new tserver with Master: a0bb9773d54d429ab9df067ab6d53913 (127.9.91.66:37823)
I20260809 19:24:39.009372 3077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:33727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:24:39.023391 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36281
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:39.101147 3619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:39.101315 3619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:39.101336 3619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:39.102389 3619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:39.102420 3619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:39.102444 3619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:39.103801 3619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:36281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:39.103962 3619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:39.104135 3619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:39.108587 3632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.108703 3629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:39.108736 3619 server_base.cc:1034] running on GCE node
W20260809 19:24:39.108706 3630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:39.108976 3619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:39.109210 3619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:39.110355 3619 hybrid_clock.cc:648] HybridClock initialized: now 1786303479110343 us; error 32 us; skew 500 ppm
Aug 09 19:24:39 dist-test-slave-1vmg krb5kdc[3005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303479, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:39.112700 3619 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:24:39.113052 3619 webserver.cc:460] Webserver started at http://127.9.91.67:41549/ using document root <none> and password file <none>
I20260809 19:24:39.113209 3619 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:39.113246 3619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:39.113329 3619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:39.114224 3619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "1b3eafc0874f4b1ca5926b95f289f1aa"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "8b48ed7872749fb9d83b71fa542a902c"
server_key_iv: "a97010c2097a23a1dda67ee5435b7902"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.114500 3619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "1b3eafc0874f4b1ca5926b95f289f1aa"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "8b48ed7872749fb9d83b71fa542a902c"
server_key_iv: "a97010c2097a23a1dda67ee5435b7902"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.115803 3619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:24:39.116432 3642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.116576 3619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:39.116647 3619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "1b3eafc0874f4b1ca5926b95f289f1aa"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "8b48ed7872749fb9d83b71fa542a902c"
server_key_iv: "a97010c2097a23a1dda67ee5435b7902"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.116714 3619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:39.126013 3619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:39.127637 3619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:39.127753 3619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:39.127941 3619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:39.128198 3619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:39.128232 3619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.128249 3619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:39.128271 3619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.133630 3619 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:32915
I20260809 19:24:39.133701 3822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:32915 every 8 connection(s)
I20260809 19:24:39.133921 3619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:24:39.137399 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 3619
I20260809 19:24:39.137506 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:24:39.137693 9581 external_mini_cluster.cc:1428] Setting key a162c752585eb593f2115bd07e00ba06
Aug 09 19:24:39 dist-test-slave-1vmg krb5kdc[3005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303479, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:39.142392 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.134520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:47915 (local address 127.9.91.126:36281)
0809 19:24:39.134567 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.134569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.134757 (+ 188us) server_negotiation.cc:407] Connection header received
0809 19:24:39.135214 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.135215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.135225 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.135272 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.136084 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.136614 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.137657 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.137924 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.137982 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.139961 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.139962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.139962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:39.139969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.141183 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:39.141534 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:39.141543 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:39.141543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:39.141597 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:39.141807 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:39.141808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:39.141808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:39.141919 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.141975 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.142225 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.142304 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:24:39.142515 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.134492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36281 (local address 127.9.91.67:47915)
0809 19:24:39.134656 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.134687 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.135133 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.135317 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.135320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.135823 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:39.136005 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.136009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.136687 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.136687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.137530 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.137534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.137996 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.137997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.138135 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.138577 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:39.138586 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:39.139874 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:39.141256 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:39.141263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:39.141265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:39.141467 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:39.141650 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:39.141651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:39.141651 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:39.141751 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:39.141973 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:39.141980 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:39.142109 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.142191 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.142331 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260809 19:24:39.142903 3823 heartbeater.cc:347] Connected to a master server at 127.9.91.126:36281
I20260809 19:24:39.143014 3823 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:39.143182 3823 heartbeater.cc:511] Master 127.9.91.126:36281 requested a full tablet report, sending...
I20260809 19:24:39.143482 3077 ts_manager.cc:194] Registered new tserver with Master: 1b3eafc0874f4b1ca5926b95f289f1aa (127.9.91.67:32915)
I20260809 19:24:39.143839 3077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:47915
I20260809 19:24:39.149225 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:24:39.155647 3174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.150461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.9.91.126:36281)
0809 19:24:39.150515 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.150518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.150605 (+ 87us) server_negotiation.cc:407] Connection header received
0809 19:24:39.150665 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.150666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.150675 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.150719 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.151309 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.151757 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.152456 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.152643 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.152694 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.153664 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.153665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.153665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:39.153672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.154636 (+ 964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:39.154933 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:39.154934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:39.154935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:39.154984 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:39.155153 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:39.155154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:39.155154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:39.155287 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.155371 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.155491 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.155569 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:24:39.156913 3077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:24:39.158481 3077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:24:39.164178 3841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.160351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:32915 (local address 127.0.0.1:53442)
0809 19:24:39.160674 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.160683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.160728 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.161084 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.161085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.161087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.161122 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.161123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.162345 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.162346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.163278 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.163281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.163969 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.163970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.164023 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.164026 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.164080 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.164111 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":42,"thread_start_us":60,"threads_started":1}
I20260809 19:24:39.164202 3844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.160520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39916 (local address 127.9.91.65:38315)
0809 19:24:39.160814 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.160818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.160825 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:39.160856 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.160857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.160867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.160931 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.161050 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.161778 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.162900 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.163806 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.163879 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.163931 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.164077 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.164122 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":78,"threads_started":1}
I20260809 19:24:39.164453 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.160506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53442 (local address 127.9.91.67:32915)
0809 19:24:39.160957 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.160961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.160970 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:39.161000 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.161002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.161015 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.161088 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.161180 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.162234 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.163350 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.163828 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.163887 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.163939 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.164372 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.164402 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":79,"threads_started":1}
I20260809 19:24:39.164178 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.160323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38315 (local address 127.0.0.1:39916)
0809 19:24:39.160645 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.160653 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.160689 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.160945 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.160946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.160948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.160990 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.160992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.161868 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.161869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.162810 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.162813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.163891 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.163892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.163971 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.163984 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.164051 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.164105 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":1,"thread_start_us":47,"threads_started":1}
I20260809 19:24:39.164644 3836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.160323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:37823 (local address 127.0.0.1:41948)
0809 19:24:39.160463 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.160496 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.160554 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.160850 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.160853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.160855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.160918 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.160919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.162441 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.162442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.163395 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.163397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.164413 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.164414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.164477 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.164480 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.164525 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.164577 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":47,"threads_started":1}
I20260809 19:24:39.164824 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.160494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.9.91.66:37823)
0809 19:24:39.160705 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.160709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.160719 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:24:39.160756 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.160758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.160771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.160835 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.161002 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.162344 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.163471 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.164141 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.164205 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.164258 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.164714 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.164750 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
I20260809 19:24:39.165185 3721 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd61d9fe47747ab950f76fa76401acb (DEFAULT_TABLE table=test-table [id=1fe6d1d9a0014de29e1de1161af31470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:39.165472 3721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd61d9fe47747ab950f76fa76401acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:39.165408 3278 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd61d9fe47747ab950f76fa76401acb (DEFAULT_TABLE table=test-table [id=1fe6d1d9a0014de29e1de1161af31470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:39.165458 3497 tablet_service.cc:1467] Processing CreateTablet for tablet 9cd61d9fe47747ab950f76fa76401acb (DEFAULT_TABLE table=test-table [id=1fe6d1d9a0014de29e1de1161af31470]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:39.165740 3497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd61d9fe47747ab950f76fa76401acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:39.165740 3278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cd61d9fe47747ab950f76fa76401acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:39.167829 3845 tablet_bootstrap.cc:492] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Bootstrap starting.
I20260809 19:24:39.168468 3845 tablet_bootstrap.cc:654] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:39.169075 3846 tablet_bootstrap.cc:492] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Bootstrap starting.
I20260809 19:24:39.169108 3845 log.cc:824] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:39.169691 3846 tablet_bootstrap.cc:654] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:39.169822 3845 tablet_bootstrap.cc:492] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: No bootstrap required, opened a new log
I20260809 19:24:39.169884 3845 ts_tablet_manager.cc:1397] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:39.170172 3846 log.cc:824] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:39.170457 3847 tablet_bootstrap.cc:492] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631: Bootstrap starting.
I20260809 19:24:39.170863 3846 tablet_bootstrap.cc:492] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: No bootstrap required, opened a new log
I20260809 19:24:39.170962 3846 ts_tablet_manager.cc:1397] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:39.171121 3847 tablet_bootstrap.cc:654] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:39.171512 3845 raft_consensus.cc:348] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.171643 3845 raft_consensus.cc:374] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:39.171674 3845 raft_consensus.cc:729] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3eafc0874f4b1ca5926b95f289f1aa, State: Initialized, Role: FOLLOWER
I20260809 19:24:39.171682 3847 log.cc:824] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:39.171785 3845 consensus_queue.cc:260] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.172031 3845 ts_tablet_manager.cc:1428] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:39.172039 3823 heartbeater.cc:503] Master 127.9.91.126:36281 was elected leader, sending a full tablet report...
I20260809 19:24:39.172380 3847 tablet_bootstrap.cc:492] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631: No bootstrap required, opened a new log
I20260809 19:24:39.172441 3847 ts_tablet_manager.cc:1397] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:39.172920 3846 raft_consensus.cc:348] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.173079 3846 raft_consensus.cc:374] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:39.173123 3846 raft_consensus.cc:729] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0bb9773d54d429ab9df067ab6d53913, State: Initialized, Role: FOLLOWER
I20260809 19:24:39.173223 3846 consensus_queue.cc:260] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.173441 3846 ts_tablet_manager.cc:1428] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:39.173460 3608 heartbeater.cc:503] Master 127.9.91.126:36281 was elected leader, sending a full tablet report...
I20260809 19:24:39.173888 3847 raft_consensus.cc:348] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.174041 3847 raft_consensus.cc:374] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:39.174070 3847 raft_consensus.cc:729] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a69852ba0b484674aaadffde2873f631, State: Initialized, Role: FOLLOWER
I20260809 19:24:39.174172 3847 consensus_queue.cc:260] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.174435 3386 heartbeater.cc:503] Master 127.9.91.126:36281 was elected leader, sending a full tablet report...
I20260809 19:24:39.174435 3847 ts_tablet_manager.cc:1428] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260809 19:24:39.249282 3609 tablet.cc:2369] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:24:39.342700 3853 raft_consensus.cc:482] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:39.342808 3853 raft_consensus.cc:504] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.343209 3853 leader_election.cc:288] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0bb9773d54d429ab9df067ab6d53913 (127.9.91.66:37823), a69852ba0b484674aaadffde2873f631 (127.9.91.65:38315)
I20260809 19:24:39.345960 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.343443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:56669 (local address 127.9.91.66:37823)
0809 19:24:39.343495 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.343496 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.343506 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:24:39.343533 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.343534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.343546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.343608 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.343792 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.344315 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.345244 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.345629 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.345681 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.345690 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.345851 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.345894 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:24:39.346000 3844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.343495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:50719 (local address 127.9.91.65:38315)
0809 19:24:39.343547 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.343548 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.343656 (+ 108us) server_negotiation.cc:407] Connection header received
0809 19:24:39.343711 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.343712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.343727 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.343790 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.343914 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.344386 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.345284 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.345684 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.345754 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.345765 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.345892 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.345937 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:24:39.346000 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.343366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:37823 (local address 127.9.91.67:56669)
0809 19:24:39.343428 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.343438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.343485 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.343616 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.343617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.343619 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.343699 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.343702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.344389 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.344390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.345154 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.345156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.345699 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.345700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.345784 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.345788 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.345837 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.345877 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 19:24:39.346004 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.343394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38315 (local address 127.9.91.67:50719)
0809 19:24:39.343593 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.343601 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.343663 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.343791 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.343792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.343793 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.343855 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.343856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.344460 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.344460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.345190 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.345192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.345770 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.345771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.345826 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.345830 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.345877 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.345911 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":124,"threads_started":1}
I20260809 19:24:39.346387 3308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd61d9fe47747ab950f76fa76401acb" candidate_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a69852ba0b484674aaadffde2873f631" is_pre_election: true
I20260809 19:24:39.346387 3536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd61d9fe47747ab950f76fa76401acb" candidate_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bb9773d54d429ab9df067ab6d53913" is_pre_election: true
I20260809 19:24:39.346542 3308 raft_consensus.cc:2454] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b3eafc0874f4b1ca5926b95f289f1aa in term 0.
I20260809 19:24:39.346562 3536 raft_consensus.cc:2454] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b3eafc0874f4b1ca5926b95f289f1aa in term 0.
I20260809 19:24:39.346778 3643 leader_election.cc:302] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b3eafc0874f4b1ca5926b95f289f1aa, a69852ba0b484674aaadffde2873f631; no voters:
I20260809 19:24:39.346906 3853 raft_consensus.cc:2781] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:24:39.347010 3853 raft_consensus.cc:482] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:24:39.347040 3853 raft_consensus.cc:3037] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:39.347728 3853 raft_consensus.cc:504] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.347847 3853 leader_election.cc:288] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [CANDIDATE]: Term 1 election: Requested vote from peers a0bb9773d54d429ab9df067ab6d53913 (127.9.91.66:37823), a69852ba0b484674aaadffde2873f631 (127.9.91.65:38315)
I20260809 19:24:39.347990 3536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd61d9fe47747ab950f76fa76401acb" candidate_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0bb9773d54d429ab9df067ab6d53913"
I20260809 19:24:39.348043 3308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cd61d9fe47747ab950f76fa76401acb" candidate_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a69852ba0b484674aaadffde2873f631"
I20260809 19:24:39.348066 3536 raft_consensus.cc:3037] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:39.348131 3308 raft_consensus.cc:3037] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:39.348757 3536 raft_consensus.cc:2454] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b3eafc0874f4b1ca5926b95f289f1aa in term 1.
I20260809 19:24:39.348917 3308 raft_consensus.cc:2454] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b3eafc0874f4b1ca5926b95f289f1aa in term 1.
I20260809 19:24:39.348944 3645 leader_election.cc:302] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b3eafc0874f4b1ca5926b95f289f1aa, a0bb9773d54d429ab9df067ab6d53913; no voters:
I20260809 19:24:39.349020 3853 raft_consensus.cc:2781] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:39.349169 3853 raft_consensus.cc:686] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 1 LEADER]: Becoming Leader. State: Replica: 1b3eafc0874f4b1ca5926b95f289f1aa, State: Running, Role: LEADER
I20260809 19:24:39.349262 3853 consensus_queue.cc:237] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } }
I20260809 19:24:39.350049 3076 catalog_manager.cc:5495] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b3eafc0874f4b1ca5926b95f289f1aa (127.9.91.67). New cstate: current_term: 1 leader_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b3eafc0874f4b1ca5926b95f289f1aa" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32915 } health_report { overall_health: HEALTHY } } }
W20260809 19:24:39.353473 3387 tablet.cc:2369] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:24:39.384387 3826 tablet.cc:2369] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:24:39.389925 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.387821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.9.91.67:32915)
0809 19:24:39.387869 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.387871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.387879 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:39.387932 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.387933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.387951 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.388008 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:39.388118 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.388544 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.389244 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.389424 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.389476 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.389624 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:39.389716 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:24:39.389784 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.389820 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.389840 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 19:24:39.391414 3536 raft_consensus.cc:1261] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 1 FOLLOWER]: Refusing update from remote peer 1b3eafc0874f4b1ca5926b95f289f1aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:39.391419 3308 raft_consensus.cc:1261] T 9cd61d9fe47747ab950f76fa76401acb P a69852ba0b484674aaadffde2873f631 [term 1 FOLLOWER]: Refusing update from remote peer 1b3eafc0874f4b1ca5926b95f289f1aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:39.391712 3853 consensus_queue.cc:1035] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0bb9773d54d429ab9df067ab6d53913" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 37823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:39.391840 3858 consensus_queue.cc:1035] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a69852ba0b484674aaadffde2873f631" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:39.396308 3077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:39.396414 3077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:39.396778 3866 mvcc.cc:205] Tried to move back new op lower bound from 7316699051584929792 to 7316699051415224320. Current Snapshot: MvccSnapshot[applied={T|T < 7316699051584929792 or (T in {7316699051584929792})}]
I20260809 19:24:39.397150 3077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8ec862c455a4eae80ff6e61db260854: Sending DeleteTablet for 3 replicas of tablet 9cd61d9fe47747ab950f76fa76401acb
I20260809 19:24:39.397490 3497 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd61d9fe47747ab950f76fa76401acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:39 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:41948
I20260809 19:24:39.397486 3721 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd61d9fe47747ab950f76fa76401acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:39 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:53442
I20260809 19:24:39.397711 3278 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cd61d9fe47747ab950f76fa76401acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:39 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:39916
I20260809 19:24:39.397976 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 3181
I20260809 19:24:39.404945 3885 tablet_replica.cc:330] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: stopping tablet replica
I20260809 19:24:39.405063 3885 raft_consensus.cc:2229] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:39.405175 3885 raft_consensus.cc:2258] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:39.397986 3886 tablet_replica.cc:330] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: stopping tablet replica
I20260809 19:24:39.405395 3886 raft_consensus.cc:2229] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:24:39.405500 3886 raft_consensus.cc:2258] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:39.406029 3886 ts_tablet_manager.cc:1905] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:39.406047 3885 ts_tablet_manager.cc:1905] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:39.407742 3886 ts_tablet_manager.cc:1918] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:39.407742 3885 ts_tablet_manager.cc:1918] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:39.407821 3885 log.cc:1196] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/9cd61d9fe47747ab950f76fa76401acb
I20260809 19:24:39.407897 3886 log.cc:1196] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/9cd61d9fe47747ab950f76fa76401acb
I20260809 19:24:39.408197 3885 ts_tablet_manager.cc:1939] T 9cd61d9fe47747ab950f76fa76401acb P a0bb9773d54d429ab9df067ab6d53913: Deleting consensus metadata
I20260809 19:24:39.408286 3886 ts_tablet_manager.cc:1939] T 9cd61d9fe47747ab950f76fa76401acb P 1b3eafc0874f4b1ca5926b95f289f1aa: Deleting consensus metadata
I20260809 19:24:39.408679 3058 catalog_manager.cc:4841] TS a0bb9773d54d429ab9df067ab6d53913 (127.9.91.66:37823): tablet 9cd61d9fe47747ab950f76fa76401acb (table test-table [id=1fe6d1d9a0014de29e1de1161af31470]) successfully deleted
I20260809 19:24:39.408704 3057 catalog_manager.cc:4841] TS 1b3eafc0874f4b1ca5926b95f289f1aa (127.9.91.67:32915): tablet 9cd61d9fe47747ab950f76fa76401acb (table test-table [id=1fe6d1d9a0014de29e1de1161af31470]) successfully deleted
W20260809 19:24:39.413412 3055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:38315 (error 108)
I20260809 19:24:39.413883 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.413761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38315 (local address 127.0.0.1:39932)
0809 19:24:39.413813 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.413853 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:38315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 19:24:39.413985 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 3393
W20260809 19:24:39.417830 3055 catalog_manager.cc:4584] TS a69852ba0b484674aaadffde2873f631 (127.9.91.65:38315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cd61d9fe47747ab950f76fa76401acb: Network error: Client connection negotiation failed: client connection to 127.9.91.65:38315: connect: Connection refused (error 111)
I20260809 19:24:39.426227 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 3619
I20260809 19:24:39.438521 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 3027
2026-08-09T19:24:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2176 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T19:24:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:39Z Disabled control of system clock
I20260809 19:24:39.469851 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38145
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:38145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:39.549216 3895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:39.549408 3895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:39.549438 3895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:39.550530 3895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:39.550568 3895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:39.550585 3895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:39.550599 3895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:39.550616 3895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:39.551950 3895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:38145
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:39.552184 3895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:39.552436 3895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:39.556730 3908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.556737 3904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.556766 3906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:39.557103 3895 server_base.cc:1034] running on GCE node
I20260809 19:24:39.557333 3895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:39.557610 3895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:39.558744 3895 hybrid_clock.cc:648] HybridClock initialized: now 1786303479558726 us; error 32 us; skew 500 ppm
I20260809 19:24:39.559965 3895 webserver.cc:460] Webserver started at http://127.9.91.126:45343/ using document root <none> and password file <none>
I20260809 19:24:39.560220 3895 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:39.560267 3895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:39.560406 3895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:39.561399 3895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "a30dca8a13074ab289b50e6d1846fb48"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "b26e2c87a3a125d88f513c4226253d82"
server_key_iv: "d7785a850abff8771ff367f1e81da9b6"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.561736 3895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "a30dca8a13074ab289b50e6d1846fb48"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "b26e2c87a3a125d88f513c4226253d82"
server_key_iv: "d7785a850abff8771ff367f1e81da9b6"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.563022 3895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:24:39.563819 3915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.563978 3895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:39.564072 3895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "a30dca8a13074ab289b50e6d1846fb48"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "b26e2c87a3a125d88f513c4226253d82"
server_key_iv: "d7785a850abff8771ff367f1e81da9b6"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.564145 3895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:39.572023 3895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:39.572268 3895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:39.572394 3895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:39.576272 3895 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:38145
I20260809 19:24:39.576306 3993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:38145 every 8 connection(s)
I20260809 19:24:39.576591 3895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:39.577406 3995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:39.579396 3995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48: Bootstrap starting.
I20260809 19:24:39.579922 3995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:39.580193 3995 log.cc:824] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:39.580796 3995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48: No bootstrap required, opened a new log
I20260809 19:24:39.582257 3995 raft_consensus.cc:348] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30dca8a13074ab289b50e6d1846fb48" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38145 } }
I20260809 19:24:39.582376 3995 raft_consensus.cc:374] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:39.582408 3995 raft_consensus.cc:729] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a30dca8a13074ab289b50e6d1846fb48, State: Initialized, Role: FOLLOWER
I20260809 19:24:39.582541 3995 consensus_queue.cc:260] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30dca8a13074ab289b50e6d1846fb48" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38145 } }
I20260809 19:24:39.582602 3995 raft_consensus.cc:388] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:39.582623 3995 raft_consensus.cc:482] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:39.582667 3995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:39.583165 3995 raft_consensus.cc:504] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30dca8a13074ab289b50e6d1846fb48" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38145 } }
I20260809 19:24:39.583249 3995 leader_election.cc:302] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a30dca8a13074ab289b50e6d1846fb48; no voters:
I20260809 19:24:39.583392 3995 leader_election.cc:288] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:39.583441 3999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:39.583611 3995 sys_catalog.cc:564] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:39.583655 3999 raft_consensus.cc:686] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [term 1 LEADER]: Becoming Leader. State: Replica: a30dca8a13074ab289b50e6d1846fb48, State: Running, Role: LEADER
I20260809 19:24:39.583760 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 3895
I20260809 19:24:39.583819 3999 consensus_queue.cc:237] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30dca8a13074ab289b50e6d1846fb48" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38145 } }
I20260809 19:24:39.583858 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:39.584053 9581 external_mini_cluster.cc:1428] Setting key 984406ad898b0ff2a57b16680c0f17a8
I20260809 19:24:39.584236 4001 sys_catalog.cc:455] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a30dca8a13074ab289b50e6d1846fb48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30dca8a13074ab289b50e6d1846fb48" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38145 } } }
I20260809 19:24:39.584317 4001 sys_catalog.cc:458] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:39.584555 4007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:39.584856 4003 sys_catalog.cc:455] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a30dca8a13074ab289b50e6d1846fb48. Latest consensus state: current_term: 1 leader_uuid: "a30dca8a13074ab289b50e6d1846fb48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a30dca8a13074ab289b50e6d1846fb48" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38145 } } }
I20260809 19:24:39.584959 4003 sys_catalog.cc:458] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:39.585180 4007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:39.586642 4007 catalog_manager.cc:1347] Generated new cluster ID: 2548d75574d148bbb5926e226baadb1a
I20260809 19:24:39.586689 4007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:39.590276 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.584516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.9.91.126:38145)
0809 19:24:39.585234 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.585243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.585271 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:24:39.585464 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.585470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.585535 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.585661 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.585879 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.587231 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.588271 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.588862 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.588988 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.589620 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.589624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.589625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:39.589816 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.589854 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:39.589868 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.589937 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.589992 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.590076 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":92,"threads_started":1}
I20260809 19:24:39.596834 4007 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:39.597488 4007 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:39.605931 4007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48: Generated new TSK 0
I20260809 19:24:39.606096 4007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:39.609102 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38145
--builtin_ntp_servers=127.9.91.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:39.691154 4027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:39.691339 4027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:39.691370 4027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:39.692461 4027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:39.692503 4027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:39.692539 4027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:39.693838 4027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:39.694073 4027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:39.694350 4027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:39.700057 4045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.700196 4047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.700070 4042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:39.700294 4027 server_base.cc:1034] running on GCE node
I20260809 19:24:39.700544 4027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:39.700778 4027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:39.701918 4027 hybrid_clock.cc:648] HybridClock initialized: now 1786303479701902 us; error 31 us; skew 500 ppm
I20260809 19:24:39.703191 4027 webserver.cc:460] Webserver started at http://127.9.91.65:32921/ using document root <none> and password file <none>
I20260809 19:24:39.703429 4027 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:39.703481 4027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:39.703613 4027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:39.704720 4027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "78836f619cf74e4a8749254702100051"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "a3341c825491a2a1e01287400e8d6b4b"
server_key_iv: "104212662b5d1c2a199a3f080c1a5797"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.705068 4027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "78836f619cf74e4a8749254702100051"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "a3341c825491a2a1e01287400e8d6b4b"
server_key_iv: "104212662b5d1c2a199a3f080c1a5797"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.706346 4027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:24:39.707108 4054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.707280 4027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:39.707360 4027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "78836f619cf74e4a8749254702100051"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "a3341c825491a2a1e01287400e8d6b4b"
server_key_iv: "104212662b5d1c2a199a3f080c1a5797"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.707432 4027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:39.719666 4027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:39.719938 4027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:39.720081 4027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:39.720294 4027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:39.720600 4027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:39.720638 4027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.720667 4027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:39.720695 4027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.726835 4027 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:33851
I20260809 19:24:39.726891 4221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:33851 every 8 connection(s)
I20260809 19:24:39.727234 4027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:24:39.732295 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.727910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:33869 (local address 127.9.91.126:38145)
0809 19:24:39.727958 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.727960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.728164 (+ 204us) server_negotiation.cc:407] Connection header received
0809 19:24:39.728693 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.728695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.728713 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.728773 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.729427 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.729936 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.730940 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.731159 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.731219 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.732010 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.732012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.732012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:39.732017 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.732053 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:39.732059 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.732103 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.732181 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.732217 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:24:39.732467 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.727870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38145 (local address 127.9.91.65:33869)
0809 19:24:39.728049 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.728081 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.728587 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.728840 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.728844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.728855 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:39.729331 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.729336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.730041 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.730042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.730823 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.730826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.731254 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.731255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.731381 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.731861 (+ 480us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:39.731862 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:39.731889 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:39.731912 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:39.732111 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:39.732117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:39.732119 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.732179 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.732278 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260809 19:24:39.732928 4224 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38145
I20260809 19:24:39.733016 4224 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:39.733184 4224 heartbeater.cc:511] Master 127.9.91.126:38145 requested a full tablet report, sending...
I20260809 19:24:39.733652 3941 ts_manager.cc:194] Registered new tserver with Master: 78836f619cf74e4a8749254702100051 (127.9.91.65:33851)
I20260809 19:24:39.734241 3941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.65:33869
I20260809 19:24:39.734467 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 4027
I20260809 19:24:39.734575 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:24:39.734754 9581 external_mini_cluster.cc:1428] Setting key 891e36a87ebb888bca38ad6a24a74161
I20260809 19:24:39.735595 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38145
--builtin_ntp_servers=127.9.91.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:39.814349 4227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:39.814522 4227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:39.814540 4227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:39.815603 4227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:39.815634 4227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:39.815656 4227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:39.816944 4227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:39.817117 4227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:39.817335 4227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:39.825930 4237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.826179 4233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.826359 4232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:39.826686 4227 server_base.cc:1034] running on GCE node
I20260809 19:24:39.826894 4227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:39.827183 4227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:39.828334 4227 hybrid_clock.cc:648] HybridClock initialized: now 1786303479828320 us; error 34 us; skew 500 ppm
I20260809 19:24:39.829418 4227 webserver.cc:460] Webserver started at http://127.9.91.66:45153/ using document root <none> and password file <none>
I20260809 19:24:39.829613 4227 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:39.829655 4227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:39.829778 4227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:39.830744 4227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "4dc2c706087a454db298bf8248a725f3"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "7b06c7dd00d9c2aa71150da189a28554"
server_key_iv: "567c5321482934d8bdd49da094c871cd"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.831091 4227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "4dc2c706087a454db298bf8248a725f3"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "7b06c7dd00d9c2aa71150da189a28554"
server_key_iv: "567c5321482934d8bdd49da094c871cd"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.832243 4227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:39.832859 4246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.833003 4227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:39.833070 4227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "4dc2c706087a454db298bf8248a725f3"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "7b06c7dd00d9c2aa71150da189a28554"
server_key_iv: "567c5321482934d8bdd49da094c871cd"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.833135 4227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:39.842463 4227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:39.842694 4227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:39.842806 4227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:39.843024 4227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:39.843300 4227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:39.843333 4227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.843351 4227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:39.843374 4227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.848801 4227 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:43533
I20260809 19:24:39.848865 4417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:43533 every 8 connection(s)
I20260809 19:24:39.849118 4227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:24:39.850451 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 4227
I20260809 19:24:39.850553 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:24:39.850728 9581 external_mini_cluster.cc:1428] Setting key 512cedf72af3e8805b3f278ba388af7e
I20260809 19:24:39.851701 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38145
--builtin_ntp_servers=127.9.91.84:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 19:24:39.854178 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.849725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:53203 (local address 127.9.91.126:38145)
0809 19:24:39.849768 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.849770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.849970 (+ 200us) server_negotiation.cc:407] Connection header received
0809 19:24:39.850402 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.850402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.850415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.850466 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.851160 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.851691 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.852930 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.853129 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.853177 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.853928 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.853930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.853930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:39.853935 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.853954 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:39.853959 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.854030 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.854072 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.854101 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:24:39.854301 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.849695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38145 (local address 127.9.91.66:53203)
0809 19:24:39.849876 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.849903 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.850309 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.850526 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.850530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.850540 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:39.851086 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.851091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.851777 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.851779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.852834 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.852837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.853203 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.853204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.853356 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.853809 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:39.853809 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:39.853833 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:39.853853 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:39.854013 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:39.854015 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:39.854017 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.854072 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.854141 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260809 19:24:39.854688 4418 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38145
I20260809 19:24:39.854771 4418 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:39.854962 4418 heartbeater.cc:511] Master 127.9.91.126:38145 requested a full tablet report, sending...
I20260809 19:24:39.855325 3941 ts_manager.cc:194] Registered new tserver with Master: 4dc2c706087a454db298bf8248a725f3 (127.9.91.66:43533)
I20260809 19:24:39.855675 3941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.66:53203
W20260809 19:24:39.932236 4421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:39.932416 4421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:39.932444 4421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:39.933503 4421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:39.933545 4421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:39.933578 4421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:39.934854 4421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:39.935097 4421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:39.935381 4421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:39.939915 4427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:39.939965 4432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:39.940110 4421 server_base.cc:1034] running on GCE node
W20260809 19:24:39.940133 4428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:39.940352 4421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:39.940564 4421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:39.941697 4421 hybrid_clock.cc:648] HybridClock initialized: now 1786303479941685 us; error 28 us; skew 500 ppm
I20260809 19:24:39.942766 4421 webserver.cc:460] Webserver started at http://127.9.91.67:42085/ using document root <none> and password file <none>
I20260809 19:24:39.943032 4421 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:39.943084 4421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:39.943210 4421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:39.944142 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "54ce20adf0164138af5902ef8391b637"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "cc88638ea16a1ea02462b2c4ec902df9"
server_key_iv: "808cc975a423b8dc38ed37bb5a5a18d9"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.944455 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "54ce20adf0164138af5902ef8391b637"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "cc88638ea16a1ea02462b2c4ec902df9"
server_key_iv: "808cc975a423b8dc38ed37bb5a5a18d9"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.945588 4421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:24:39.946264 4442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.946421 4421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:39.946499 4421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "54ce20adf0164138af5902ef8391b637"
format_stamp: "Formatted at 2026-08-09 19:24:39 on dist-test-slave-1vmg"
server_key: "cc88638ea16a1ea02462b2c4ec902df9"
server_key_iv: "808cc975a423b8dc38ed37bb5a5a18d9"
server_key_version: "encryptionkey@0"
I20260809 19:24:39.946563 4421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:39.954011 4421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:39.954239 4421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:39.954355 4421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:39.954531 4421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:39.954805 4421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:39.954843 4421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.954869 4421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:39.954892 4421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:39.960282 4421 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:42775
I20260809 19:24:39.960335 4608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:42775 every 8 connection(s)
I20260809 19:24:39.960608 4421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:24:39.965174 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.961206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:56761 (local address 127.9.91.126:38145)
0809 19:24:39.961281 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.961284 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.961437 (+ 153us) server_negotiation.cc:407] Connection header received
0809 19:24:39.961869 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.961870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.961884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.961933 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.962535 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.963111 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.964008 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.964213 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.964268 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.964907 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.964908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.964908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:39.964913 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.964928 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:39.964933 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.964977 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.965063 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.965095 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 19:24:39.965330 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.961171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38145 (local address 127.9.91.67:56761)
0809 19:24:39.961338 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.961367 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.961756 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.961992 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.961995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.962004 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:39.962448 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.962451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.963188 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.963189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.963922 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.963924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.964282 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.964283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.964398 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.964783 (+ 385us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:39.964783 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:39.964807 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:39.964828 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:39.964984 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:39.964989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:39.964991 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.965060 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.965154 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260809 19:24:39.965749 4609 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38145
I20260809 19:24:39.965832 4609 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:39.965996 4609 heartbeater.cc:511] Master 127.9.91.126:38145 requested a full tablet report, sending...
I20260809 19:24:39.966372 3941 ts_manager.cc:194] Registered new tserver with Master: 54ce20adf0164138af5902ef8391b637 (127.9.91.67:42775)
I20260809 19:24:39.966717 3941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.67:56761
I20260809 19:24:39.967226 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 4421
I20260809 19:24:39.967327 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:24:39.967505 9581 external_mini_cluster.cc:1428] Setting key e6a249a48b40348a0e4898eec6ba07d3
I20260809 19:24:39.968304 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:24:39.971859 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.969534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.9.91.126:38145)
0809 19:24:39.969577 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.969579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.969696 (+ 117us) server_negotiation.cc:407] Connection header received
0809 19:24:39.969721 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.969722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.969733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.969775 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:39.969893 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.970318 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.970994 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.971174 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.971219 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.971677 (+ 458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:39.971678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:39.971678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:39.971681 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:39.971691 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:39.971695 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:39.971732 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.971771 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.971790 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:24:39.973078 3941 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:24:39.974485 3941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:24:39.980229 4624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.976352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:42775 (local address 127.0.0.1:59226)
0809 19:24:39.976653 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.976661 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.976717 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.976888 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.976891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.976893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.976948 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.976949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.977919 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.977920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.978992 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.978995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.979960 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.979961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.980042 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.980055 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.980115 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.980161 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":5,"thread_start_us":85,"threads_started":1}
I20260809 19:24:39.980389 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.976436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.9.91.66:43533)
0809 19:24:39.976974 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.976978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.976986 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:39.977019 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.977021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.977034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.977107 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.977249 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.978319 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.979555 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.980063 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.980124 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.980205 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.980274 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.980312 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":73,"threads_started":1}
I20260809 19:24:39.980515 4618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.976335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:43533 (local address 127.0.0.1:60614)
0809 19:24:39.976609 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.976646 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.976712 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.977122 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.977123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.977125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.977166 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.977167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.978402 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.978403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.979458 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.979461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.980142 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.980143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.980204 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.980208 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.980262 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.980404 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":73,"threads_started":1}
I20260809 19:24:39.980541 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.976426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.9.91.67:42775)
0809 19:24:39.976555 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.976559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.976718 (+ 159us) server_negotiation.cc:407] Connection header received
0809 19:24:39.976793 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.976795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.976808 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.976870 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.977004 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.977845 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.979084 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.979788 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.979852 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.979927 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.980420 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.980456 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260809 19:24:39.981005 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.976544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:33851 (local address 127.0.0.1:44782)
0809 19:24:39.977202 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.977212 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.977258 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.977507 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.977508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.977510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.977565 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.977566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.978666 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.978667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.979560 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.979562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.980604 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.980605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.980845 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.980849 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.980909 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.980945 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260809 19:24:39.981045 4626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.976757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44782 (local address 127.9.91.65:33851)
0809 19:24:39.976923 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.976927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.977317 (+ 390us) server_negotiation.cc:407] Connection header received
0809 19:24:39.977362 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.977364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.977379 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.977445 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.977635 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.978599 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.979634 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.980417 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.980661 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.980751 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.980932 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.980971 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260809 19:24:39.981145 4506 tablet_service.cc:1467] Processing CreateTablet for tablet 335e6db925d84a2c81a5b6f7fa277809 (DEFAULT_TABLE table=test-table [id=eb3a965f232c43b794ef1db6b477c8ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:39.981189 4313 tablet_service.cc:1467] Processing CreateTablet for tablet 335e6db925d84a2c81a5b6f7fa277809 (DEFAULT_TABLE table=test-table [id=eb3a965f232c43b794ef1db6b477c8ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:39.981492 4506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335e6db925d84a2c81a5b6f7fa277809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:39.981528 4313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335e6db925d84a2c81a5b6f7fa277809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:39.981613 4121 tablet_service.cc:1467] Processing CreateTablet for tablet 335e6db925d84a2c81a5b6f7fa277809 (DEFAULT_TABLE table=test-table [id=eb3a965f232c43b794ef1db6b477c8ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:39.981896 4121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 335e6db925d84a2c81a5b6f7fa277809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:39.983862 4627 tablet_bootstrap.cc:492] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Bootstrap starting.
I20260809 19:24:39.984695 4627 tablet_bootstrap.cc:654] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:39.985010 4629 tablet_bootstrap.cc:492] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051: Bootstrap starting.
I20260809 19:24:39.985008 4627 log.cc:824] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:39.985641 4629 tablet_bootstrap.cc:654] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:39.985777 4627 tablet_bootstrap.cc:492] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: No bootstrap required, opened a new log
I20260809 19:24:39.985846 4627 ts_tablet_manager.cc:1397] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:39.985975 4629 log.cc:824] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:39.986303 4628 tablet_bootstrap.cc:492] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Bootstrap starting.
I20260809 19:24:39.986629 4629 tablet_bootstrap.cc:492] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051: No bootstrap required, opened a new log
I20260809 19:24:39.986686 4629 ts_tablet_manager.cc:1397] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:39.986956 4628 tablet_bootstrap.cc:654] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:39.987268 4628 log.cc:824] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:39.988173 4627 raft_consensus.cc:348] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.988288 4627 raft_consensus.cc:374] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:39.988317 4627 raft_consensus.cc:729] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ce20adf0164138af5902ef8391b637, State: Initialized, Role: FOLLOWER
I20260809 19:24:39.988411 4627 consensus_queue.cc:260] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.988641 4627 ts_tablet_manager.cc:1428] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:24:39.988713 4609 heartbeater.cc:503] Master 127.9.91.126:38145 was elected leader, sending a full tablet report...
I20260809 19:24:39.988672 4629 raft_consensus.cc:348] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.988816 4629 raft_consensus.cc:374] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:39.988853 4629 raft_consensus.cc:729] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78836f619cf74e4a8749254702100051, State: Initialized, Role: FOLLOWER
I20260809 19:24:39.988955 4629 consensus_queue.cc:260] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.989199 4629 ts_tablet_manager.cc:1428] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:24:39.989226 4224 heartbeater.cc:503] Master 127.9.91.126:38145 was elected leader, sending a full tablet report...
I20260809 19:24:39.989313 4628 tablet_bootstrap.cc:492] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: No bootstrap required, opened a new log
I20260809 19:24:39.989379 4628 ts_tablet_manager.cc:1397] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 19:24:39.989708 4634 raft_consensus.cc:482] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:39.989796 4634 raft_consensus.cc:504] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.990302 4634 leader_election.cc:288] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dc2c706087a454db298bf8248a725f3 (127.9.91.66:43533), 78836f619cf74e4a8749254702100051 (127.9.91.65:33851)
I20260809 19:24:39.991866 4628 raft_consensus.cc:348] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.992004 4628 raft_consensus.cc:374] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:39.992041 4628 raft_consensus.cc:729] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc2c706087a454db298bf8248a725f3, State: Initialized, Role: FOLLOWER
I20260809 19:24:39.992136 4628 consensus_queue.cc:260] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.992342 4628 ts_tablet_manager.cc:1428] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:24:39.992399 4418 heartbeater.cc:503] Master 127.9.91.126:38145 was elected leader, sending a full tablet report...
I20260809 19:24:39.993145 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.990304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:46237 (local address 127.9.91.66:43533)
0809 19:24:39.990337 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.990339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.990404 (+ 65us) server_negotiation.cc:407] Connection header received
0809 19:24:39.990453 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.990453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.990462 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.990510 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.990711 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.991192 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.992216 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.992874 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.992932 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.992938 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.993063 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.993093 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:24:39.993166 4638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.990176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:43533 (local address 127.9.91.67:46237)
0809 19:24:39.990348 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.990357 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.990409 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.990622 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.990622 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.990623 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.990660 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.990661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.991265 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.991266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.992144 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.992147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.992934 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.992935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.993005 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.993008 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.993052 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.993085 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":8,"thread_start_us":76,"threads_started":1}
I20260809 19:24:39.993271 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.990176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:33851 (local address 127.9.91.67:51353)
0809 19:24:39.990214 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:39.990223 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:39.990256 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:39.990511 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:39.990511 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:39.990512 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:39.990565 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.990566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.991357 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.991358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.992370 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:39.992373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.993052 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.993053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:39.993126 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.993128 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:39.993171 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:24:39.993204 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 19:24:39.993461 4344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335e6db925d84a2c81a5b6f7fa277809" candidate_uuid: "54ce20adf0164138af5902ef8391b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc2c706087a454db298bf8248a725f3" is_pre_election: true
I20260809 19:24:39.993567 4626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.990255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:51353 (local address 127.9.91.65:33851)
0809 19:24:39.990415 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.990416 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.990423 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:39.990440 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.990441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.990450 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.990510 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:39.990611 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.991259 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.992510 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.992976 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.993038 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:39.993046 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:39.993181 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 19:24:39.993507 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 19:24:39.993590 4344 raft_consensus.cc:2454] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ce20adf0164138af5902ef8391b637 in term 0.
I20260809 19:24:39.993768 4154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335e6db925d84a2c81a5b6f7fa277809" candidate_uuid: "54ce20adf0164138af5902ef8391b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78836f619cf74e4a8749254702100051" is_pre_election: true
I20260809 19:24:39.993772 4445 leader_election.cc:302] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc2c706087a454db298bf8248a725f3, 54ce20adf0164138af5902ef8391b637; no voters:
I20260809 19:24:39.993852 4154 raft_consensus.cc:2454] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ce20adf0164138af5902ef8391b637 in term 0.
I20260809 19:24:39.993880 4634 raft_consensus.cc:2781] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:24:39.993922 4634 raft_consensus.cc:482] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:24:39.993948 4634 raft_consensus.cc:3037] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:39.994467 4634 raft_consensus.cc:504] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.994585 4634 leader_election.cc:288] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [CANDIDATE]: Term 1 election: Requested vote from peers 4dc2c706087a454db298bf8248a725f3 (127.9.91.66:43533), 78836f619cf74e4a8749254702100051 (127.9.91.65:33851)
I20260809 19:24:39.994700 4344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335e6db925d84a2c81a5b6f7fa277809" candidate_uuid: "54ce20adf0164138af5902ef8391b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc2c706087a454db298bf8248a725f3"
I20260809 19:24:39.994710 4154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "335e6db925d84a2c81a5b6f7fa277809" candidate_uuid: "54ce20adf0164138af5902ef8391b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78836f619cf74e4a8749254702100051"
I20260809 19:24:39.994768 4344 raft_consensus.cc:3037] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:39.994769 4154 raft_consensus.cc:3037] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:39.995375 4154 raft_consensus.cc:2454] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ce20adf0164138af5902ef8391b637 in term 1.
I20260809 19:24:39.995390 4344 raft_consensus.cc:2454] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ce20adf0164138af5902ef8391b637 in term 1.
I20260809 19:24:39.995523 4444 leader_election.cc:302] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ce20adf0164138af5902ef8391b637, 78836f619cf74e4a8749254702100051; no voters:
I20260809 19:24:39.995621 4634 raft_consensus.cc:2781] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:39.995752 4634 raft_consensus.cc:686] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 1 LEADER]: Becoming Leader. State: Replica: 54ce20adf0164138af5902ef8391b637, State: Running, Role: LEADER
I20260809 19:24:39.995824 4634 consensus_queue.cc:237] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } }
I20260809 19:24:39.996439 3941 catalog_manager.cc:5495] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ce20adf0164138af5902ef8391b637 (127.9.91.67). New cstate: current_term: 1 leader_uuid: "54ce20adf0164138af5902ef8391b637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ce20adf0164138af5902ef8391b637" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 42775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 } health_report { overall_health: UNKNOWN } } }
I20260809 19:24:40.000209 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:39.998430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.9.91.67:42775)
0809 19:24:39.998461 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:39.998463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:39.998470 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:39.998535 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:39.998535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:39.998547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:39.998590 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:39.998655 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.999094 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.999724 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:39.999895 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:39.999933 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:40.000015 (+ 82us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:40.000087 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:24:40.000133 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:40.000145 (+ 12us) server_negotiation.cc:299] Negotiation successful
0809 19:24:40.000156 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:24:40.001502 4154 raft_consensus.cc:1261] T 335e6db925d84a2c81a5b6f7fa277809 P 78836f619cf74e4a8749254702100051 [term 1 FOLLOWER]: Refusing update from remote peer 54ce20adf0164138af5902ef8391b637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:40.001549 4344 raft_consensus.cc:1261] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 1 FOLLOWER]: Refusing update from remote peer 54ce20adf0164138af5902ef8391b637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:40.001791 4642 consensus_queue.cc:1035] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dc2c706087a454db298bf8248a725f3" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 43533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:40.001899 4634 consensus_queue.cc:1035] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78836f619cf74e4a8749254702100051" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:40.003859 4647 mvcc.cc:205] Tried to move back new op lower bound from 7316699054084288512 to 7316699054063497216. Current Snapshot: MvccSnapshot[applied={T|T < 7316699054084288512}]
I20260809 19:24:40.004038 4646 mvcc.cc:205] Tried to move back new op lower bound from 7316699054084288512 to 7316699054063497216. Current Snapshot: MvccSnapshot[applied={T|T < 7316699054084288512}]
I20260809 19:24:40.006457 3941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:40.006543 3941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:40.007143 3941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a30dca8a13074ab289b50e6d1846fb48: Sending DeleteTablet for 3 replicas of tablet 335e6db925d84a2c81a5b6f7fa277809
I20260809 19:24:40.007429 4313 tablet_service.cc:1514] Processing DeleteTablet for tablet 335e6db925d84a2c81a5b6f7fa277809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:40 UTC) from {username='slave'} at 127.0.0.1:60614
I20260809 19:24:40.007475 4506 tablet_service.cc:1514] Processing DeleteTablet for tablet 335e6db925d84a2c81a5b6f7fa277809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:40 UTC) from {username='slave'} at 127.0.0.1:59226
I20260809 19:24:40.007462 4121 tablet_service.cc:1514] Processing DeleteTablet for tablet 335e6db925d84a2c81a5b6f7fa277809 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:40 UTC) from {username='slave'} at 127.0.0.1:44782
I20260809 19:24:40.007702 4659 tablet_replica.cc:330] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: stopping tablet replica
I20260809 19:24:40.007714 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 4027
I20260809 19:24:40.023031 4658 tablet_replica.cc:330] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: stopping tablet replica
I20260809 19:24:40.023167 4658 raft_consensus.cc:2229] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:40.023303 4658 raft_consensus.cc:2258] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:40.026100 4658 ts_tablet_manager.cc:1905] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:40.027953 4658 ts_tablet_manager.cc:1918] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:40.028045 4658 log.cc:1196] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/335e6db925d84a2c81a5b6f7fa277809
I20260809 19:24:40.028375 4658 ts_tablet_manager.cc:1939] T 335e6db925d84a2c81a5b6f7fa277809 P 4dc2c706087a454db298bf8248a725f3: Deleting consensus metadata
I20260809 19:24:40.028856 3920 catalog_manager.cc:4841] TS 4dc2c706087a454db298bf8248a725f3 (127.9.91.66:43533): tablet 335e6db925d84a2c81a5b6f7fa277809 (table test-table [id=eb3a965f232c43b794ef1db6b477c8ad]) successfully deleted
I20260809 19:24:40.038944 4659 raft_consensus.cc:2229] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:24:40.039105 4659 raft_consensus.cc:2258] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:40.039515 4659 ts_tablet_manager.cc:1905] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:40.040795 4659 ts_tablet_manager.cc:1918] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:40.040858 4659 log.cc:1196] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/335e6db925d84a2c81a5b6f7fa277809
I20260809 19:24:40.041085 4659 ts_tablet_manager.cc:1939] T 335e6db925d84a2c81a5b6f7fa277809 P 54ce20adf0164138af5902ef8391b637: Deleting consensus metadata
I20260809 19:24:40.041416 3919 catalog_manager.cc:4841] TS 54ce20adf0164138af5902ef8391b637 (127.9.91.67:42775): tablet 335e6db925d84a2c81a5b6f7fa277809 (table test-table [id=eb3a965f232c43b794ef1db6b477c8ad]) successfully deleted
W20260809 19:24:40.045660 3919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:33851 (error 108)
I20260809 19:24:40.045920 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 4227
I20260809 19:24:40.047437 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:40.045987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:33851 (local address 127.0.0.1:44792)
0809 19:24:40.047358 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:40.047405 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:33851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1347}
W20260809 19:24:40.048424 3919 catalog_manager.cc:4584] TS 78836f619cf74e4a8749254702100051 (127.9.91.65:33851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 335e6db925d84a2c81a5b6f7fa277809: Network error: Client connection negotiation failed: client connection to 127.9.91.65:33851: connect: Connection refused (error 111)
I20260809 19:24:40.054183 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 4421
I20260809 19:24:40.066087 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 3895
2026-08-09T19:24:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (626 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:40 dist-test-slave-1vmg krb5kdc[4670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:40 dist-test-slave-1vmg krb5kdc[4670](info): set up 2 sockets
Aug 09 19:24:40 dist-test-slave-1vmg krb5kdc[4670](info): commencing operation
krb5kdc: starting...
W20260809 19:24:41.194459 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:24:41.301365 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43057
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:43057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:41.381366 4693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:41.381539 4693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:41.381559 4693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:41.382603 4693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:41.382635 4693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:41.382648 4693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:41.382659 4693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:41.382673 4693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:41.384084 4693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:43057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:43057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:41.384258 4693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:41.384415 4693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:41.388855 4704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:41.389012 4693 server_base.cc:1034] running on GCE node
W20260809 19:24:41.388875 4699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:41.388863 4700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:41.389353 4693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:41.389611 4693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:41.390738 4693 hybrid_clock.cc:648] HybridClock initialized: now 1786303481390728 us; error 28 us; skew 500 ppm
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:41.392985 4693 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:24:41.393394 4693 webserver.cc:460] Webserver started at http://127.9.91.126:39121/ using document root <none> and password file <none>
I20260809 19:24:41.393577 4693 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:41.393622 4693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:41.393743 4693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:41.394682 4693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "35ddbf7f59044e32b1c94eac6b217f8b"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "f2ca61b7c03f79883e2d322769a6d2b4"
server_key_iv: "408bd1512774bb93afd46502e9ee2241"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.395032 4693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "35ddbf7f59044e32b1c94eac6b217f8b"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "f2ca61b7c03f79883e2d322769a6d2b4"
server_key_iv: "408bd1512774bb93afd46502e9ee2241"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.396205 4693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:41.396801 4713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.396944 4693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:41.397024 4693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "35ddbf7f59044e32b1c94eac6b217f8b"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "f2ca61b7c03f79883e2d322769a6d2b4"
server_key_iv: "408bd1512774bb93afd46502e9ee2241"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.397096 4693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:41.420045 4693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:41.421597 4693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:41.421720 4693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:41.425482 4693 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:43057
I20260809 19:24:41.425530 4792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:43057 every 8 connection(s)
I20260809 19:24:41.425798 4693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:41.426551 4797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:41.428587 4797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b: Bootstrap starting.
I20260809 19:24:41.429119 4797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:41.429374 4797 log.cc:824] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:41.429978 4797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b: No bootstrap required, opened a new log
I20260809 19:24:41.431639 4797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43057 } }
I20260809 19:24:41.431771 4797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:41.431794 4797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ddbf7f59044e32b1c94eac6b217f8b, State: Initialized, Role: FOLLOWER
I20260809 19:24:41.431926 4797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43057 } }
I20260809 19:24:41.431995 4797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:41.432029 4797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:41.432075 4797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:41.432555 4797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43057 } }
I20260809 19:24:41.432646 4797 leader_election.cc:302] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35ddbf7f59044e32b1c94eac6b217f8b; no voters:
I20260809 19:24:41.432808 4797 leader_election.cc:288] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:41.432842 4802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:41.432981 4802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [term 1 LEADER]: Becoming Leader. State: Replica: 35ddbf7f59044e32b1c94eac6b217f8b, State: Running, Role: LEADER
I20260809 19:24:41.433027 4797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:41.433081 4802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43057 } }
I20260809 19:24:41.433550 4804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43057 } } }
I20260809 19:24:41.433629 4804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:41.433738 4807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ddbf7f59044e32b1c94eac6b217f8b. Latest consensus state: current_term: 1 leader_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ddbf7f59044e32b1c94eac6b217f8b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 43057 } } }
I20260809 19:24:41.433811 4807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:41.434028 4816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:41.434542 4816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:41.435230 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 4693
I20260809 19:24:41.435339 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:41.435556 9581 external_mini_cluster.cc:1428] Setting key d8e04b9dea1553a21407180d438cf89e
I20260809 19:24:41.436096 4816 catalog_manager.cc:1347] Generated new cluster ID: 28b296b1b0884d3cb7eff20ea5f893df
I20260809 19:24:41.436149 4816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:41.449645 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.435964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.9.91.126:43057)
0809 19:24:41.436128 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.436142 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.439131 (+ 2989us) server_negotiation.cc:407] Connection header received
0809 19:24:41.439269 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.439275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.439326 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.439463 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:41.440246 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.441900 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.442997 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.443562 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.443654 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.446217 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:41.446221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:41.446221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:41.446232 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:41.448064 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.448408 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.448411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.448411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.448519 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.448687 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.448687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.448688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.448984 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:41.449133 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.449199 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.449428 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260809 19:24:41.479182 4816 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:41.479856 4816 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:41.493387 4816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b: Generated new TSK 0
I20260809 19:24:41.493546 4816 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:24:41.522899 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43057
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:41.601246 4841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:41.601426 4841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:41.601455 4841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:41.602519 4841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:41.602560 4841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:41.602594 4841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:41.604020 4841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:41.604262 4841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:41.604454 4841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:41.609113 4849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:41.609120 4847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:41.609120 4851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:41.609174 4841 server_base.cc:1034] running on GCE node
I20260809 19:24:41.609475 4841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:41.609673 4841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:41.610806 4841 hybrid_clock.cc:648] HybridClock initialized: now 1786303481610794 us; error 31 us; skew 500 ppm
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:41.613209 4841 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:24:41.613631 4841 webserver.cc:460] Webserver started at http://127.9.91.65:33975/ using document root <none> and password file <none>
I20260809 19:24:41.613827 4841 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:41.613874 4841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:41.613997 4841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:41.614979 4841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "be8867bf3a0b49e2b929778e87fe1e5c"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "bd08028d74a7e2b265cc1fa93950a54d"
server_key_iv: "846c5ff6cf2afd9de0e8e209d8695ff5"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.615304 4841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "be8867bf3a0b49e2b929778e87fe1e5c"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "bd08028d74a7e2b265cc1fa93950a54d"
server_key_iv: "846c5ff6cf2afd9de0e8e209d8695ff5"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.616483 4841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:41.617151 4861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.617290 4841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:41.617372 4841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "be8867bf3a0b49e2b929778e87fe1e5c"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "bd08028d74a7e2b265cc1fa93950a54d"
server_key_iv: "846c5ff6cf2afd9de0e8e209d8695ff5"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.617437 4841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:41.628484 4841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:41.630385 4841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:41.630513 4841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:41.630704 4841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:41.631006 4841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:41.631048 4841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.631074 4841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:41.631093 4841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.636806 4841 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:46679
I20260809 19:24:41.636847 5027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:46679 every 8 connection(s)
I20260809 19:24:41.637120 4841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:41.644937 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.637737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35013 (local address 127.9.91.126:43057)
0809 19:24:41.637781 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.637783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.637953 (+ 170us) server_negotiation.cc:407] Connection header received
0809 19:24:41.638392 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.638393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.638407 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.638456 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:41.639352 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.639869 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.640760 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.640950 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.640991 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.642845 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:41.642847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:41.642847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:41.642854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:41.643931 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.644266 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.644267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.644267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.644320 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.644477 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.644477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.644477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.644571 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:41.644618 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.644807 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.644874 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:24:41.645073 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.637695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43057 (local address 127.9.91.65:35013)
0809 19:24:41.637848 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:41.637885 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:41.638319 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:41.638504 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:41.638507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:41.639094 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:41.639269 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.639273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.639932 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.639933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.640699 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.640701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.640999 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.641000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.641097 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.641501 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:41.641508 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:41.642761 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:41.643996 (+ 1235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:41.644003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:41.644004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:41.644207 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:41.644360 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:41.644360 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:41.644361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:41.644440 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:41.644614 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:41.644616 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:41.644716 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 19:24:41.644789 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 19:24:41.644898 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260809 19:24:41.645540 5028 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43057
I20260809 19:24:41.645627 5028 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:41.645762 5028 heartbeater.cc:511] Master 127.9.91.126:43057 requested a full tablet report, sending...
I20260809 19:24:41.646240 4736 ts_manager.cc:194] Registered new tserver with Master: be8867bf3a0b49e2b929778e87fe1e5c (127.9.91.65:46679)
I20260809 19:24:41.646837 4736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:35013
I20260809 19:24:41.647300 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 4841
I20260809 19:24:41.647413 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:24:41.647620 9581 external_mini_cluster.cc:1428] Setting key 972228a75e8dc8984fe63583137a8f67
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:24:41.670212 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43057
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:41.750005 5033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:41.750187 5033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:41.750209 5033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:41.751314 5033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:41.751353 5033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:41.751379 5033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:41.752739 5033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:41.752908 5033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:41.753070 5033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:41.757807 5041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:41.757841 5044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:41.757936 5033 server_base.cc:1034] running on GCE node
W20260809 19:24:41.757851 5047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:41.758211 5033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:41.758446 5033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:41.759622 5033 hybrid_clock.cc:648] HybridClock initialized: now 1786303481759600 us; error 35 us; skew 500 ppm
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:41.761978 5033 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:24:41.762367 5033 webserver.cc:460] Webserver started at http://127.9.91.66:35693/ using document root <none> and password file <none>
I20260809 19:24:41.762560 5033 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:41.762607 5033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:41.762730 5033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:41.763725 5033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "a849e845f92c44bc8ed9983f890ba067"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "e56dfd308b159b429528be5672cd5735"
server_key_iv: "176c085a8f17015585e743cf88392829"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.764055 5033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "a849e845f92c44bc8ed9983f890ba067"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "e56dfd308b159b429528be5672cd5735"
server_key_iv: "176c085a8f17015585e743cf88392829"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.765267 5033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:24:41.765894 5058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.766043 5033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:41.766124 5033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "a849e845f92c44bc8ed9983f890ba067"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "e56dfd308b159b429528be5672cd5735"
server_key_iv: "176c085a8f17015585e743cf88392829"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.766186 5033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:41.791795 5033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:41.793493 5033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:41.793630 5033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:41.793838 5033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:41.794147 5033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:41.794180 5033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.794207 5033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:41.794231 5033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.800429 5033 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:41281
I20260809 19:24:41.800463 5231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:41281 every 8 connection(s)
I20260809 19:24:41.800786 5033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:24:41.804505 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 5033
I20260809 19:24:41.804615 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:24:41.804849 9581 external_mini_cluster.cc:1428] Setting key cf47d71aa13fb168bf02947c58e77d1f
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:41.809751 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.801443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:42799 (local address 127.9.91.126:43057)
0809 19:24:41.801491 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.801494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.801705 (+ 211us) server_negotiation.cc:407] Connection header received
0809 19:24:41.802166 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.802167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.802178 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.802237 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:41.803140 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.803645 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.804730 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.805023 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.805099 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.807237 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:41.807239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:41.807239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:41.807247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:41.808466 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.808888 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.808888 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.808889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.808942 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.809188 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.809188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.809188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.809287 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:41.809334 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.809628 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.809689 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:24:41.810014 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.801409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43057 (local address 127.9.91.66:42799)
0809 19:24:41.801583 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:41.801622 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:41.802089 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:41.802288 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:41.802291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:41.802831 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:41.803066 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.803072 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.803778 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.803779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.804640 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.804644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.805115 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.805117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.805250 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.805716 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:41.805726 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:41.807151 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:41.808548 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:41.808555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:41.808557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:41.808820 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:41.808998 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:41.808998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:41.808999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:41.809131 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:41.809342 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:41.809348 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:41.809500 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 19:24:41.809618 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 19:24:41.809771 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260809 19:24:41.810503 5232 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43057
I20260809 19:24:41.810616 5232 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:41.810801 5232 heartbeater.cc:511] Master 127.9.91.126:43057 requested a full tablet report, sending...
I20260809 19:24:41.811213 4736 ts_manager.cc:194] Registered new tserver with Master: a849e845f92c44bc8ed9983f890ba067 (127.9.91.66:41281)
I20260809 19:24:41.811710 4736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:42799
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:24:41.829068 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43057
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:33733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:41.908161 5239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:41.908351 5239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:41.908372 5239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:41.909466 5239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:41.909503 5239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:41.909528 5239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:41.910946 5239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:33733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:43057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:41.911177 5239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:41.911386 5239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:41.915966 5246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:41.916033 5250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:41.916042 5247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:41.916225 5239 server_base.cc:1034] running on GCE node
I20260809 19:24:41.916427 5239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:41.916719 5239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:41.917872 5239 hybrid_clock.cc:648] HybridClock initialized: now 1786303481917850 us; error 41 us; skew 500 ppm
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:41.920998 5239 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:24:41.921409 5239 webserver.cc:460] Webserver started at http://127.9.91.67:34097/ using document root <none> and password file <none>
I20260809 19:24:41.921653 5239 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:41.921703 5239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:41.921849 5239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:41.922883 5239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "fb201458a93f46669a3c636aa9f5d952"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "c07938c4bdc78855e3d85d4f92a2f81b"
server_key_iv: "bd8445246c6d7a6e74cc44e8c33fbe9c"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.923290 5239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "fb201458a93f46669a3c636aa9f5d952"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "c07938c4bdc78855e3d85d4f92a2f81b"
server_key_iv: "bd8445246c6d7a6e74cc44e8c33fbe9c"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.924656 5239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:41.925350 5262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.925505 5239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:41.925585 5239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "fb201458a93f46669a3c636aa9f5d952"
format_stamp: "Formatted at 2026-08-09 19:24:41 on dist-test-slave-1vmg"
server_key: "c07938c4bdc78855e3d85d4f92a2f81b"
server_key_iv: "bd8445246c6d7a6e74cc44e8c33fbe9c"
server_key_version: "encryptionkey@0"
I20260809 19:24:41.925652 5239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:41.940939 5239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:41.942581 5239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:41.942705 5239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:41.942895 5239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:41.943269 5239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:41.943303 5239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.943329 5239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:41.943353 5239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:41.948746 5239 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:37183
I20260809 19:24:41.948813 5440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:37183 every 8 connection(s)
I20260809 19:24:41.949065 5239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:24:41.953397 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 5239
I20260809 19:24:41.953505 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:24:41.953709 9581 external_mini_cluster.cc:1428] Setting key ea5312ee97eda27fc9f27765b888d231
Aug 09 19:24:41 dist-test-slave-1vmg krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303481, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:41.957381 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.949706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:50927 (local address 127.9.91.126:43057)
0809 19:24:41.949767 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.949770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.949942 (+ 172us) server_negotiation.cc:407] Connection header received
0809 19:24:41.950351 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.950352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.950363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.950433 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:41.951302 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.951797 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.952773 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.952957 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.953011 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.955199 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:41.955200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:41.955200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:41.955207 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:41.956434 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.956742 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.956742 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.956743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.956787 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.956944 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.956945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.956945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.957039 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:41.957083 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.957267 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.957320 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:24:41.957551 5447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.949678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:43057 (local address 127.9.91.67:50927)
0809 19:24:41.949835 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:41.949868 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:41.950277 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:41.950480 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:41.950483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:41.951057 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:41.951230 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.951235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.951905 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.951906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.952692 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.952694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.953014 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.953015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.953103 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.953529 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:41.953539 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:41.955108 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:41.956491 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:41.956493 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:41.956495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:41.956690 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:41.956827 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:41.956827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:41.956827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:41.956907 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:41.957081 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:41.957083 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:41.957182 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 19:24:41.957253 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:24:41.957361 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260809 19:24:41.957949 5445 heartbeater.cc:347] Connected to a master server at 127.9.91.126:43057
I20260809 19:24:41.958033 5445 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:41.958194 5445 heartbeater.cc:511] Master 127.9.91.126:43057 requested a full tablet report, sending...
I20260809 19:24:41.958531 4736 ts_manager.cc:194] Registered new tserver with Master: fb201458a93f46669a3c636aa9f5d952 (127.9.91.67:37183)
I20260809 19:24:41.958874 4736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:50927
I20260809 19:24:41.965337 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:24:41.971704 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.966514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41028 (local address 127.9.91.126:43057)
0809 19:24:41.966575 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.966577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.966636 (+ 59us) server_negotiation.cc:407] Connection header received
0809 19:24:41.966683 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.966683 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.966691 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.966733 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:41.967247 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.967682 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.968318 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.968517 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.968563 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.969695 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:41.969696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:41.969696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:41.969702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:41.970728 (+ 1026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.971125 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.971125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.971125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.971174 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:41.971313 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:41.971314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:41.971314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:41.971419 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:41.971484 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.971578 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.971626 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 19:24:41.973021 4736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:24:41.974490 4736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:24:41.980257 5457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.976512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:37183 (local address 127.0.0.1:58270)
0809 19:24:41.976747 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:41.976785 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:41.976874 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:41.977131 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:41.977134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:41.977135 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:41.977195 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.977197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.978459 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.978460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.979294 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.979297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.979982 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.979982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.980057 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.980069 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 19:24:41.980140 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:24:41.980189 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":27,"thread_start_us":63,"threads_started":1}
I20260809 19:24:41.980263 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.976582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58270 (local address 127.9.91.67:37183)
0809 19:24:41.976742 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.976746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.976850 (+ 104us) server_negotiation.cc:407] Connection header received
0809 19:24:41.976965 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.976967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.976982 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.977045 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:41.977274 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.978341 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.979370 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.979907 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.979975 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.980027 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.980156 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.980191 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260809 19:24:41.980530 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.976699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:46679 (local address 127.0.0.1:45534)
0809 19:24:41.976894 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:41.976903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:41.976954 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:41.977200 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:41.977201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:41.977203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:41.977247 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.977249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.978232 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.978233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.979428 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.979430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.980328 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.980329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.980397 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.980403 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 19:24:41.980430 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 19:24:41.980469 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":3,"thread_start_us":37,"threads_started":1}
I20260809 19:24:41.980868 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.976893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45534 (local address 127.9.91.65:46679)
0809 19:24:41.977056 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.977059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.977068 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:41.977107 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.977109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.977121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.977187 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:41.977352 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.978137 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.979515 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.980233 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.980677 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.980745 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.980769 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.980805 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 19:24:41.981040 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.976512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41281 (local address 127.0.0.1:36682)
0809 19:24:41.976791 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:41.976800 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:41.976855 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:41.977130 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:41.977133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:41.977135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:41.977200 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.977201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.978456 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.978457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.979766 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:41.979769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.980805 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.980806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:41.980869 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.980873 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:41.980925 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 19:24:41.980961 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":7,"thread_start_us":51,"threads_started":1}
I20260809 19:24:41.981050 5340 tablet_service.cc:1467] Processing CreateTablet for tablet 387f29a060a7442e8d0f02e889d5fabe (DEFAULT_TABLE table=test-table [id=38b125cec457429e8fac11cc47c551f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:41.981324 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:41.976618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.9.91.66:41281)
0809 19:24:41.976836 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:41.976840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:41.976851 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:24:41.976966 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:41.976969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:41.976982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:41.977049 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:41.977298 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.978367 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.979847 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:41.980654 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:41.980720 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:41.980771 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:41.981072 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 19:24:41.981239 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":92,"threads_started":1}
I20260809 19:24:41.981376 5340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387f29a060a7442e8d0f02e889d5fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:41.981474 4925 tablet_service.cc:1467] Processing CreateTablet for tablet 387f29a060a7442e8d0f02e889d5fabe (DEFAULT_TABLE table=test-table [id=38b125cec457429e8fac11cc47c551f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:41.981796 4925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387f29a060a7442e8d0f02e889d5fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:41.983934 5463 tablet_bootstrap.cc:492] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Bootstrap starting.
I20260809 19:24:41.984450 5133 tablet_service.cc:1467] Processing CreateTablet for tablet 387f29a060a7442e8d0f02e889d5fabe (DEFAULT_TABLE table=test-table [id=38b125cec457429e8fac11cc47c551f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:41.984696 5463 tablet_bootstrap.cc:654] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:41.984787 5133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 387f29a060a7442e8d0f02e889d5fabe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:41.985232 5463 log.cc:824] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:41.985997 5463 tablet_bootstrap.cc:492] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: No bootstrap required, opened a new log
I20260809 19:24:41.986085 5463 ts_tablet_manager.cc:1397] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:41.987092 5464 tablet_bootstrap.cc:492] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c: Bootstrap starting.
I20260809 19:24:41.987619 5464 tablet_bootstrap.cc:654] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:41.987960 5463 raft_consensus.cc:348] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:41.988147 5463 raft_consensus.cc:374] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:41.988148 5464 log.cc:824] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:41.988198 5463 raft_consensus.cc:729] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb201458a93f46669a3c636aa9f5d952, State: Initialized, Role: FOLLOWER
I20260809 19:24:41.988319 5466 tablet_bootstrap.cc:492] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Bootstrap starting.
I20260809 19:24:41.988330 5463 consensus_queue.cc:260] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:41.988595 5463 ts_tablet_manager.cc:1428] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:24:41.988623 5445 heartbeater.cc:503] Master 127.9.91.126:43057 was elected leader, sending a full tablet report...
I20260809 19:24:41.989054 5464 tablet_bootstrap.cc:492] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c: No bootstrap required, opened a new log
I20260809 19:24:41.989077 5466 tablet_bootstrap.cc:654] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:41.989112 5464 ts_tablet_manager.cc:1397] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:41.989611 5466 log.cc:824] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:41.990279 5466 tablet_bootstrap.cc:492] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: No bootstrap required, opened a new log
I20260809 19:24:41.990338 5466 ts_tablet_manager.cc:1397] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:41.990716 5464 raft_consensus.cc:348] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:41.990837 5464 raft_consensus.cc:374] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:41.990867 5464 raft_consensus.cc:729] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be8867bf3a0b49e2b929778e87fe1e5c, State: Initialized, Role: FOLLOWER
I20260809 19:24:41.990991 5464 consensus_queue.cc:260] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:41.991252 5464 ts_tablet_manager.cc:1428] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:41.991277 5028 heartbeater.cc:503] Master 127.9.91.126:43057 was elected leader, sending a full tablet report...
I20260809 19:24:41.992219 5466 raft_consensus.cc:348] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:41.992410 5466 raft_consensus.cc:374] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:41.992458 5466 raft_consensus.cc:729] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a849e845f92c44bc8ed9983f890ba067, State: Initialized, Role: FOLLOWER
I20260809 19:24:41.992554 5466 consensus_queue.cc:260] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:41.992765 5466 ts_tablet_manager.cc:1428] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:24:41.992800 5232 heartbeater.cc:503] Master 127.9.91.126:43057 was elected leader, sending a full tablet report...
W20260809 19:24:42.051352 5233 tablet.cc:2369] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:24:42.137773 5029 tablet.cc:2369] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:24:42.158438 5470 raft_consensus.cc:482] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:42.158545 5470 raft_consensus.cc:504] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:42.158845 5470 leader_election.cc:288] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a849e845f92c44bc8ed9983f890ba067 (127.9.91.66:41281), be8867bf3a0b49e2b929778e87fe1e5c (127.9.91.65:46679)
I20260809 19:24:42.161608 5461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.159217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:43033 (local address 127.9.91.65:46679)
0809 19:24:42.159267 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.159269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.159392 (+ 123us) server_negotiation.cc:407] Connection header received
0809 19:24:42.159421 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.159422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.159434 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.159477 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:42.159608 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.160073 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.160955 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.161309 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.161367 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.161377 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.161502 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.161544 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:24:42.161616 5459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.159125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:52819 (local address 127.9.91.66:41281)
0809 19:24:42.159176 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.159178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.159187 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:42.159215 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.159215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.159227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.159286 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:42.159447 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.159996 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.160906 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.161302 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.161358 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.161369 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.161508 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.161551 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:24:42.161617 5447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.159034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:41281 (local address 127.9.91.67:52819)
0809 19:24:42.159083 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.159095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.159136 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.159296 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.159297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.159299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:42.159366 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.159367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.160069 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.160070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.160822 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.160824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.161372 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.161373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.161447 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.161449 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.161496 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.161536 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:24:42.161624 5478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.159158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:46679 (local address 127.9.91.67:43033)
0809 19:24:42.159332 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.159340 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.159384 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.159489 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.159489 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.159491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:42.159554 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.159556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.160141 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.160141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.160876 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.160878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.161380 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.161380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.161444 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.161447 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.161491 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.161530 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260809 19:24:42.161962 5165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387f29a060a7442e8d0f02e889d5fabe" candidate_uuid: "fb201458a93f46669a3c636aa9f5d952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a849e845f92c44bc8ed9983f890ba067" is_pre_election: true
I20260809 19:24:42.162084 5165 raft_consensus.cc:2454] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb201458a93f46669a3c636aa9f5d952 in term 0.
I20260809 19:24:42.162292 5263 leader_election.cc:302] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a849e845f92c44bc8ed9983f890ba067, fb201458a93f46669a3c636aa9f5d952; no voters:
I20260809 19:24:42.162384 4956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387f29a060a7442e8d0f02e889d5fabe" candidate_uuid: "fb201458a93f46669a3c636aa9f5d952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" is_pre_election: true
I20260809 19:24:42.162420 5470 raft_consensus.cc:2781] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:24:42.162485 5470 raft_consensus.cc:482] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:24:42.162518 5470 raft_consensus.cc:3037] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:42.162520 4956 raft_consensus.cc:2454] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb201458a93f46669a3c636aa9f5d952 in term 0.
I20260809 19:24:42.163323 5470 raft_consensus.cc:504] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:42.163436 5470 leader_election.cc:288] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [CANDIDATE]: Term 1 election: Requested vote from peers a849e845f92c44bc8ed9983f890ba067 (127.9.91.66:41281), be8867bf3a0b49e2b929778e87fe1e5c (127.9.91.65:46679)
I20260809 19:24:42.163614 5165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387f29a060a7442e8d0f02e889d5fabe" candidate_uuid: "fb201458a93f46669a3c636aa9f5d952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a849e845f92c44bc8ed9983f890ba067"
I20260809 19:24:42.163676 4956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "387f29a060a7442e8d0f02e889d5fabe" candidate_uuid: "fb201458a93f46669a3c636aa9f5d952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be8867bf3a0b49e2b929778e87fe1e5c"
I20260809 19:24:42.163702 5165 raft_consensus.cc:3037] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:42.163745 4956 raft_consensus.cc:3037] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:42.164598 5165 raft_consensus.cc:2454] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb201458a93f46669a3c636aa9f5d952 in term 1.
I20260809 19:24:42.164598 4956 raft_consensus.cc:2454] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb201458a93f46669a3c636aa9f5d952 in term 1.
I20260809 19:24:42.164754 5263 leader_election.cc:302] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be8867bf3a0b49e2b929778e87fe1e5c, fb201458a93f46669a3c636aa9f5d952; no voters:
I20260809 19:24:42.164839 5470 raft_consensus.cc:2781] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:42.164956 5470 raft_consensus.cc:686] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 1 LEADER]: Becoming Leader. State: Replica: fb201458a93f46669a3c636aa9f5d952, State: Running, Role: LEADER
I20260809 19:24:42.165045 5470 consensus_queue.cc:237] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } }
I20260809 19:24:42.165781 4736 catalog_manager.cc:5495] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb201458a93f46669a3c636aa9f5d952 (127.9.91.67). New cstate: current_term: 1 leader_uuid: "fb201458a93f46669a3c636aa9f5d952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb201458a93f46669a3c636aa9f5d952" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 37183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 } health_report { overall_health: UNKNOWN } } }
W20260809 19:24:42.199533 5446 tablet.cc:2369] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:24:42.204154 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.202051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.9.91.67:37183)
0809 19:24:42.202100 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.202102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.202109 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:42.202163 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.202163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.202180 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.202233 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:42.202314 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.202791 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.203536 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.203723 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.203790 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.203915 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:42.204001 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:24:42.204073 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.204096 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.204103 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:24:42.205812 5165 raft_consensus.cc:1261] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 1 FOLLOWER]: Refusing update from remote peer fb201458a93f46669a3c636aa9f5d952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:42.205812 4956 raft_consensus.cc:1261] T 387f29a060a7442e8d0f02e889d5fabe P be8867bf3a0b49e2b929778e87fe1e5c [term 1 FOLLOWER]: Refusing update from remote peer fb201458a93f46669a3c636aa9f5d952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:42.206175 5470 consensus_queue.cc:1035] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be8867bf3a0b49e2b929778e87fe1e5c" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 46679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:42.206310 5479 consensus_queue.cc:1035] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a849e845f92c44bc8ed9983f890ba067" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:42.208709 5489 mvcc.cc:205] Tried to move back new op lower bound from 7316699063112556544 to 7316699062948626432. Current Snapshot: MvccSnapshot[applied={T|T < 7316699063112556544}]
I20260809 19:24:42.208978 5486 mvcc.cc:205] Tried to move back new op lower bound from 7316699063112556544 to 7316699062948626432. Current Snapshot: MvccSnapshot[applied={T|T < 7316699063112556544 or (T in {7316699063112556544})}]
I20260809 19:24:42.209371 5487 mvcc.cc:205] Tried to move back new op lower bound from 7316699063112556544 to 7316699062948626432. Current Snapshot: MvccSnapshot[applied={T|T < 7316699063112556544}]
I20260809 19:24:42.210680 4736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:42.210790 4736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:42.211563 4736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35ddbf7f59044e32b1c94eac6b217f8b: Sending DeleteTablet for 3 replicas of tablet 387f29a060a7442e8d0f02e889d5fabe
I20260809 19:24:42.211848 5340 tablet_service.cc:1514] Processing DeleteTablet for tablet 387f29a060a7442e8d0f02e889d5fabe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:42 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:58270
I20260809 19:24:42.211905 4925 tablet_service.cc:1514] Processing DeleteTablet for tablet 387f29a060a7442e8d0f02e889d5fabe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:42 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:45534
I20260809 19:24:42.211939 5133 tablet_service.cc:1514] Processing DeleteTablet for tablet 387f29a060a7442e8d0f02e889d5fabe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:42 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:36682
I20260809 19:24:42.212148 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 4841
I20260809 19:24:42.212286 5502 tablet_replica.cc:330] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: stopping tablet replica
I20260809 19:24:42.212404 5502 raft_consensus.cc:2229] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:24:42.212568 5502 raft_consensus.cc:2258] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:42.220279 5502 ts_tablet_manager.cc:1905] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:42.221488 5504 tablet_replica.cc:330] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: stopping tablet replica
I20260809 19:24:42.221606 5504 raft_consensus.cc:2229] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:42.221709 5504 raft_consensus.cc:2258] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:42.222062 5502 ts_tablet_manager.cc:1918] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:42.222121 5502 log.cc:1196] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/387f29a060a7442e8d0f02e889d5fabe
I20260809 19:24:42.222342 5502 ts_tablet_manager.cc:1939] T 387f29a060a7442e8d0f02e889d5fabe P fb201458a93f46669a3c636aa9f5d952: Deleting consensus metadata
I20260809 19:24:42.222681 4716 catalog_manager.cc:4841] TS fb201458a93f46669a3c636aa9f5d952 (127.9.91.67:37183): tablet 387f29a060a7442e8d0f02e889d5fabe (table test-table [id=38b125cec457429e8fac11cc47c551f4]) successfully deleted
I20260809 19:24:42.223282 5504 ts_tablet_manager.cc:1905] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:42.224869 5504 ts_tablet_manager.cc:1918] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:42.224946 5504 log.cc:1196] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/387f29a060a7442e8d0f02e889d5fabe
I20260809 19:24:42.225199 5504 ts_tablet_manager.cc:1939] T 387f29a060a7442e8d0f02e889d5fabe P a849e845f92c44bc8ed9983f890ba067: Deleting consensus metadata
I20260809 19:24:42.225593 4714 catalog_manager.cc:4841] TS a849e845f92c44bc8ed9983f890ba067 (127.9.91.66:41281): tablet 387f29a060a7442e8d0f02e889d5fabe (table test-table [id=38b125cec457429e8fac11cc47c551f4]) successfully deleted
W20260809 19:24:42.225951 4714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:46679 (error 108)
I20260809 19:24:42.226233 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 5033
I20260809 19:24:42.226428 5456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.226298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:46679 (local address 127.0.0.1:45538)
0809 19:24:42.226356 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.226395 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:46679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 19:24:42.229329 4714 catalog_manager.cc:4584] TS be8867bf3a0b49e2b929778e87fe1e5c (127.9.91.65:46679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 387f29a060a7442e8d0f02e889d5fabe: Network error: Client connection negotiation failed: client connection to 127.9.91.65:46679: connect: Connection refused (error 111)
I20260809 19:24:42.239727 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 5239
I20260809 19:24:42.249626 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 4693
2026-08-09T19:24:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T19:24:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:42Z Disabled control of system clock
I20260809 19:24:42.278637 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:41525
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:35569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:41525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:42.358399 5511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:42.358590 5511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:42.358609 5511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:42.359724 5511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:42.359776 5511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:42.359787 5511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:42.359799 5511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:42.359809 5511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:42.361075 5511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:41525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:41525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:42.361246 5511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:42.361461 5511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:42.365805 5519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:42.365895 5511 server_base.cc:1034] running on GCE node
W20260809 19:24:42.365811 5520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:42.365823 5528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:42.366243 5511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:42.366525 5511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:42.367660 5511 hybrid_clock.cc:648] HybridClock initialized: now 1786303482367647 us; error 33 us; skew 500 ppm
I20260809 19:24:42.368855 5511 webserver.cc:460] Webserver started at http://127.9.91.126:44429/ using document root <none> and password file <none>
I20260809 19:24:42.369060 5511 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:42.369112 5511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:42.369242 5511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:42.370235 5511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "5a5b263c8c195eb258a86f1b63848b6c"
server_key_iv: "45bd7362469c5c33174d4d1dcf76b71e"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.370579 5511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "5a5b263c8c195eb258a86f1b63848b6c"
server_key_iv: "45bd7362469c5c33174d4d1dcf76b71e"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.371889 5511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:24:42.372588 5538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.372740 5511 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:42.372828 5511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "5a5b263c8c195eb258a86f1b63848b6c"
server_key_iv: "45bd7362469c5c33174d4d1dcf76b71e"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.372892 5511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:42.387174 5511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:42.387426 5511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:42.387568 5511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:42.391518 5511 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:41525
I20260809 19:24:42.391551 5614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:41525 every 8 connection(s)
I20260809 19:24:42.391848 5511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:42.392549 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 5511
I20260809 19:24:42.392652 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:42.392804 5616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:42.392866 9581 external_mini_cluster.cc:1428] Setting key 70710c16a63374987282453149aea146
I20260809 19:24:42.395478 5616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9: Bootstrap starting.
I20260809 19:24:42.396140 5616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:42.396445 5616 log.cc:824] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:42.397130 5616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9: No bootstrap required, opened a new log
I20260809 19:24:42.398180 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.393377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46560 (local address 127.9.91.126:41525)
0809 19:24:42.393572 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.393579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.393604 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:24:42.394036 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.394042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.394096 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.394205 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:42.394423 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.395357 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.396370 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.396788 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.396879 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.397521 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:42.397525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:42.397526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:42.397717 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:42.397757 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:42.397771 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:42.397841 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.397901 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.397975 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260809 19:24:42.399137 5616 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41525 } }
I20260809 19:24:42.399255 5616 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:42.399276 5616 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2dc4765f6341cda0bb1a5553c3e3d9, State: Initialized, Role: FOLLOWER
I20260809 19:24:42.399382 5616 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41525 } }
I20260809 19:24:42.399446 5616 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:42.399469 5616 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:42.399513 5616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:42.399998 5616 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41525 } }
I20260809 19:24:42.400106 5616 leader_election.cc:302] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea2dc4765f6341cda0bb1a5553c3e3d9; no voters:
I20260809 19:24:42.400259 5616 leader_election.cc:288] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:42.400328 5626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:42.400482 5626 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [term 1 LEADER]: Becoming Leader. State: Replica: ea2dc4765f6341cda0bb1a5553c3e3d9, State: Running, Role: LEADER
I20260809 19:24:42.400490 5616 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:42.400589 5626 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41525 } }
I20260809 19:24:42.402002 5629 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea2dc4765f6341cda0bb1a5553c3e3d9. Latest consensus state: current_term: 1 leader_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41525 } } }
I20260809 19:24:42.402097 5629 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:42.402289 5628 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea2dc4765f6341cda0bb1a5553c3e3d9" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 41525 } } }
I20260809 19:24:42.402302 5646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:42.402369 5628 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:42.403587 5646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:42.405373 5646 catalog_manager.cc:1347] Generated new cluster ID: 88d6fe39e68c4eeabcf0d381d4384ef0
I20260809 19:24:42.405434 5646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:42.456367 5646 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:42.456904 5646 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:42.460050 5646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9: Generated new TSK 0
I20260809 19:24:42.460235 5646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:42.464658 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41525
--builtin_ntp_servers=127.9.91.84:35569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:42.544984 5651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:42.545171 5651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:42.545192 5651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:42.546288 5651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:42.546327 5651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:42.546353 5651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:42.547693 5651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:42.547912 5651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:42.548172 5651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:42.552690 5658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:42.552692 5657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:42.552843 5651 server_base.cc:1034] running on GCE node
W20260809 19:24:42.552722 5661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:42.553167 5651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:42.553390 5651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:42.554531 5651 hybrid_clock.cc:648] HybridClock initialized: now 1786303482554519 us; error 29 us; skew 500 ppm
I20260809 19:24:42.555683 5651 webserver.cc:460] Webserver started at http://127.9.91.65:33941/ using document root <none> and password file <none>
I20260809 19:24:42.555881 5651 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:42.555924 5651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:42.556051 5651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:42.556999 5651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "a4bc09165a174ff99bf83e63a7ebde5d"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "2f872fe7b51a55ccb9c05d697a249ff7"
server_key_iv: "603392b710e445654c0e31c4d750c768"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.557309 5651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "a4bc09165a174ff99bf83e63a7ebde5d"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "2f872fe7b51a55ccb9c05d697a249ff7"
server_key_iv: "603392b710e445654c0e31c4d750c768"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.558449 5651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:24:42.559084 5671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.559209 5651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.559283 5651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "a4bc09165a174ff99bf83e63a7ebde5d"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "2f872fe7b51a55ccb9c05d697a249ff7"
server_key_iv: "603392b710e445654c0e31c4d750c768"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.559345 5651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:42.574270 5651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:42.574501 5651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:42.574625 5651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:42.574826 5651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:42.575183 5651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:42.575227 5651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.575251 5651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:42.575278 5651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.581281 5651 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:32947
I20260809 19:24:42.581321 5852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:32947 every 8 connection(s)
I20260809 19:24:42.581619 5651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:24:42.586018 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.582262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:33195 (local address 127.9.91.126:41525)
0809 19:24:42.582310 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.582312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.582527 (+ 215us) server_negotiation.cc:407] Connection header received
0809 19:24:42.582931 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.582932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.582948 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.583021 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:42.583601 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.584095 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.584964 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.585142 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.585189 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.585799 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:42.585800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:42.585801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:42.585805 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:42.585821 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:42.585824 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:42.585873 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.585927 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.585953 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:24:42.586158 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.582237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:41525 (local address 127.9.91.65:33195)
0809 19:24:42.582419 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.582446 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.582838 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.583066 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.583070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.583078 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:42.583515 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.583519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.584164 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.584165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.584902 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.584904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.585197 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.585198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.585297 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.585683 (+ 386us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:42.585684 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:42.585707 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:42.585728 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:42.585875 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:42.585877 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:42.585879 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.585919 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.585986 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260809 19:24:42.586639 5853 heartbeater.cc:347] Connected to a master server at 127.9.91.126:41525
I20260809 19:24:42.586714 5853 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:42.586872 5853 heartbeater.cc:511] Master 127.9.91.126:41525 requested a full tablet report, sending...
I20260809 19:24:42.587318 5559 ts_manager.cc:194] Registered new tserver with Master: a4bc09165a174ff99bf83e63a7ebde5d (127.9.91.65:32947)
I20260809 19:24:42.587885 5559 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.65:33195
I20260809 19:24:42.589994 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 5651
I20260809 19:24:42.590093 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:24:42.590263 9581 external_mini_cluster.cc:1428] Setting key 05ad05cd9f307fe693ea7743500eb5dd
I20260809 19:24:42.591104 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41525
--builtin_ntp_servers=127.9.91.84:35569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:42.669608 5862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:42.669788 5862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:42.669808 5862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:42.670892 5862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:42.670956 5862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:42.670993 5862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:42.672338 5862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:42.672510 5862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:42.672729 5862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:42.677142 5869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:42.677199 5868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:42.677150 5872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:42.677295 5862 server_base.cc:1034] running on GCE node
I20260809 19:24:42.677558 5862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:42.677784 5862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:42.678943 5862 hybrid_clock.cc:648] HybridClock initialized: now 1786303482678928 us; error 28 us; skew 500 ppm
I20260809 19:24:42.680114 5862 webserver.cc:460] Webserver started at http://127.9.91.66:40045/ using document root <none> and password file <none>
I20260809 19:24:42.680320 5862 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:42.680365 5862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:42.680483 5862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:42.681420 5862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "adac7063c3994281b46c1de705704b10"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "40348465bd4590b146866d75720a844e"
server_key_iv: "cf93d1a3b691da1f6c1339358017b2df"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.681725 5862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "adac7063c3994281b46c1de705704b10"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "40348465bd4590b146866d75720a844e"
server_key_iv: "cf93d1a3b691da1f6c1339358017b2df"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.682854 5862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:24:42.683529 5881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.683686 5862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:42.683764 5862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "adac7063c3994281b46c1de705704b10"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "40348465bd4590b146866d75720a844e"
server_key_iv: "cf93d1a3b691da1f6c1339358017b2df"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.683823 5862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:42.701170 5862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:42.701411 5862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:42.701536 5862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:42.701736 5862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:42.702044 5862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:42.702087 5862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.702117 5862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:42.702144 5862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.708114 5862 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:36429
I20260809 19:24:42.708149 6050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:36429 every 8 connection(s)
I20260809 19:24:42.708426 5862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:24:42.712842 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.709043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:51867 (local address 127.9.91.126:41525)
0809 19:24:42.709101 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.709103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.709312 (+ 209us) server_negotiation.cc:407] Connection header received
0809 19:24:42.709741 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.709742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.709753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.709793 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:42.710353 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.710832 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.711719 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.711901 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.711954 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.712593 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:42.712594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:42.712595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:42.712599 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:42.712615 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:42.712619 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:42.712660 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.712756 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.712780 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:24:42.712937 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.709010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:41525 (local address 127.9.91.66:51867)
0809 19:24:42.709201 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.709236 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.709652 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.709844 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.709847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.709855 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:42.710294 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.710298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.710899 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.710900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.711654 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.711656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.711960 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.711961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.712058 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.712467 (+ 409us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:42.712468 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:42.712492 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:42.712512 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:42.712670 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:42.712672 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:42.712674 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.712720 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.712788 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260809 19:24:42.713348 6052 heartbeater.cc:347] Connected to a master server at 127.9.91.126:41525
I20260809 19:24:42.713428 6052 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:42.713590 6052 heartbeater.cc:511] Master 127.9.91.126:41525 requested a full tablet report, sending...
I20260809 19:24:42.713920 5559 ts_manager.cc:194] Registered new tserver with Master: adac7063c3994281b46c1de705704b10 (127.9.91.66:36429)
I20260809 19:24:42.714255 5559 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.66:51867
I20260809 19:24:42.715868 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 5862
I20260809 19:24:42.715950 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:24:42.716142 9581 external_mini_cluster.cc:1428] Setting key 6a1eae4f976fba9b6cac475f5820ae64
I20260809 19:24:42.716922 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41525
--builtin_ntp_servers=127.9.91.84:35569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:42.795161 6058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:42.795326 6058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:42.795346 6058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:42.796428 6058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:42.796466 6058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:42.796489 6058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:42.797776 6058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:35569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:41525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:42.797968 6058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:42.798215 6058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:42.802667 6065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:42.802765 6067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:42.802685 6063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:42.802901 6058 server_base.cc:1034] running on GCE node
I20260809 19:24:42.803113 6058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:42.803287 6058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:42.804417 6058 hybrid_clock.cc:648] HybridClock initialized: now 1786303482804403 us; error 27 us; skew 500 ppm
I20260809 19:24:42.805500 6058 webserver.cc:460] Webserver started at http://127.9.91.67:45599/ using document root <none> and password file <none>
I20260809 19:24:42.805704 6058 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:42.805750 6058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:42.805868 6058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:42.806805 6058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "b45884d68cc84ff0a1cd5b53f89a47d2"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "fc7ff89612981e7442a51de7b6fadf89"
server_key_iv: "0bb6b79fd76e639d9aaf169fe1efe93e"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.807132 6058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "b45884d68cc84ff0a1cd5b53f89a47d2"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "fc7ff89612981e7442a51de7b6fadf89"
server_key_iv: "0bb6b79fd76e639d9aaf169fe1efe93e"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.808254 6058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:42.808872 6075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.809012 6058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.809093 6058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "b45884d68cc84ff0a1cd5b53f89a47d2"
format_stamp: "Formatted at 2026-08-09 19:24:42 on dist-test-slave-1vmg"
server_key: "fc7ff89612981e7442a51de7b6fadf89"
server_key_iv: "0bb6b79fd76e639d9aaf169fe1efe93e"
server_key_version: "encryptionkey@0"
I20260809 19:24:42.809149 6058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:42.822608 6058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:42.822825 6058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:42.822993 6058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:42.823189 6058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:42.823501 6058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:42.823544 6058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.823570 6058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:42.823594 6058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:42.828890 6058 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:32869
I20260809 19:24:42.828949 6252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:32869 every 8 connection(s)
I20260809 19:24:42.829198 6058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:24:42.831715 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 6058
I20260809 19:24:42.831808 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:24:42.831988 9581 external_mini_cluster.cc:1428] Setting key d655d2bc38b2345e688f37cd9cd0f5a3
I20260809 19:24:42.833889 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.829792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:56157 (local address 127.9.91.126:41525)
0809 19:24:42.829837 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.829839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.830046 (+ 207us) server_negotiation.cc:407] Connection header received
0809 19:24:42.830437 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.830438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.830447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.830496 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:42.831115 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.831608 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.832770 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.833024 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.833079 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.833689 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:42.833690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:42.833690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:42.833694 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:42.833707 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:42.833711 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:42.833750 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.833806 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.833826 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:24:42.834050 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.829766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:41525 (local address 127.9.91.67:56157)
0809 19:24:42.829934 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.829969 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.830368 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.830550 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.830553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.830560 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:42.831050 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.831055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.831694 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.831696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.832701 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.832704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.833090 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.833091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.833193 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.833579 (+ 386us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:42.833579 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:42.833603 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:42.833624 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:42.833756 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:42.833759 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:42.833760 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.833803 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.833870 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260809 19:24:42.834447 6253 heartbeater.cc:347] Connected to a master server at 127.9.91.126:41525
I20260809 19:24:42.834539 6253 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:42.834673 6253 heartbeater.cc:511] Master 127.9.91.126:41525 requested a full tablet report, sending...
I20260809 19:24:42.835006 5559 ts_manager.cc:194] Registered new tserver with Master: b45884d68cc84ff0a1cd5b53f89a47d2 (127.9.91.67:32869)
I20260809 19:24:42.835340 5559 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.67:56157
I20260809 19:24:42.843519 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:24:42.847378 5623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.844727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46570 (local address 127.9.91.126:41525)
0809 19:24:42.844784 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.844786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.844891 (+ 105us) server_negotiation.cc:407] Connection header received
0809 19:24:42.844939 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.844940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.844949 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.845000 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:42.845145 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.845586 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.846283 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.846472 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.846525 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.847108 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:42.847110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:42.847110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:42.847114 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:42.847127 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:42.847130 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:42.847208 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.847292 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.847302 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 19:24:42.848697 5559 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:24:42.850142 5559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:24:42.855827 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.852009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:36429 (local address 127.0.0.1:51856)
0809 19:24:42.852290 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.852300 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.852338 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.852756 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.852757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.852759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:42.852790 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.852791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.853684 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.853685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.854780 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.854783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.855563 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.855564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.855638 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.855651 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.855713 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.855763 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":94,"threads_started":1}
I20260809 19:24:42.855855 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.852447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51856 (local address 127.9.91.66:36429)
0809 19:24:42.852626 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.852630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.852639 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:42.852674 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.852676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.852691 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.852761 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:42.852840 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.853606 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.854873 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.855490 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.855553 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.855638 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.855741 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.855779 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260809 19:24:42.856165 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.851987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:32869 (local address 127.0.0.1:43256)
0809 19:24:42.852161 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.852188 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.852249 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.852801 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.852802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.852804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:42.852848 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.852849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.854091 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.854092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.855150 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.855153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.855973 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.855974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.856028 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.856032 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.856060 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.856098 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":89,"threads_started":1}
I20260809 19:24:42.856323 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.851987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:32947 (local address 127.0.0.1:59926)
0809 19:24:42.852151 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.852186 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.852248 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.852568 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.852571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.852572 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:42.852635 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.852636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.854124 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.854125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.854879 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.854881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.855785 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.855786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.855848 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.855852 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.856221 (+ 369us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.856267 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260809 19:24:42.856474 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.852148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.9.91.67:32869)
0809 19:24:42.852643 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.852647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.852655 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:42.852683 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.852685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.852697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.852927 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:42.852969 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.853997 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.855230 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.855884 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.856267 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.856354 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.856378 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.856413 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":75,"threads_started":1}
I20260809 19:24:42.856645 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.852251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59926 (local address 127.9.91.65:32947)
0809 19:24:42.852406 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.852410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.852419 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:42.852455 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.852457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.852472 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.852563 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:42.852713 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.854022 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.854999 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.855681 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.856416 (+ 735us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.856512 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.856539 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.856578 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260809 19:24:42.856523 5947 tablet_service.cc:1467] Processing CreateTablet for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 (DEFAULT_TABLE table=test-table [id=c2146fbc6cc346ba90fe4486c9f9cae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:42.856868 5947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:42.859229 6273 tablet_bootstrap.cc:492] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Bootstrap starting.
I20260809 19:24:42.859848 6273 tablet_bootstrap.cc:654] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:42.859778 6152 tablet_service.cc:1467] Processing CreateTablet for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 (DEFAULT_TABLE table=test-table [id=c2146fbc6cc346ba90fe4486c9f9cae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:42.860055 6152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:42.860044 5748 tablet_service.cc:1467] Processing CreateTablet for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 (DEFAULT_TABLE table=test-table [id=c2146fbc6cc346ba90fe4486c9f9cae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:42.860177 6273 log.cc:824] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:42.860356 5748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:42.860908 6273 tablet_bootstrap.cc:492] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: No bootstrap required, opened a new log
I20260809 19:24:42.860961 6273 ts_tablet_manager.cc:1397] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:42.862490 6278 tablet_bootstrap.cc:492] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Bootstrap starting.
I20260809 19:24:42.863131 6278 tablet_bootstrap.cc:654] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:42.863456 6278 log.cc:824] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:42.863443 6273 raft_consensus.cc:348] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.863597 6273 raft_consensus.cc:374] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:42.863637 6273 raft_consensus.cc:729] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adac7063c3994281b46c1de705704b10, State: Initialized, Role: FOLLOWER
I20260809 19:24:42.863766 6273 consensus_queue.cc:260] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.863999 6273 ts_tablet_manager.cc:1428] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:24:42.864075 6052 heartbeater.cc:503] Master 127.9.91.126:41525 was elected leader, sending a full tablet report...
I20260809 19:24:42.864189 6280 tablet_bootstrap.cc:492] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d: Bootstrap starting.
I20260809 19:24:42.864789 6280 tablet_bootstrap.cc:654] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:42.865099 6280 log.cc:824] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:42.865712 6280 tablet_bootstrap.cc:492] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d: No bootstrap required, opened a new log
I20260809 19:24:42.865734 6278 tablet_bootstrap.cc:492] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: No bootstrap required, opened a new log
I20260809 19:24:42.865763 6280 ts_tablet_manager.cc:1397] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:42.865788 6278 ts_tablet_manager.cc:1397] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 19:24:42.867473 6280 raft_consensus.cc:348] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.867595 6280 raft_consensus.cc:374] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:42.867615 6280 raft_consensus.cc:729] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4bc09165a174ff99bf83e63a7ebde5d, State: Initialized, Role: FOLLOWER
I20260809 19:24:42.867702 6280 consensus_queue.cc:260] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.867761 6278 raft_consensus.cc:348] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.867918 6280 ts_tablet_manager.cc:1428] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 19:24:42.867925 6278 raft_consensus.cc:374] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:42.867965 6278 raft_consensus.cc:729] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45884d68cc84ff0a1cd5b53f89a47d2, State: Initialized, Role: FOLLOWER
I20260809 19:24:42.867981 5853 heartbeater.cc:503] Master 127.9.91.126:41525 was elected leader, sending a full tablet report...
I20260809 19:24:42.868047 6278 consensus_queue.cc:260] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.868243 6278 ts_tablet_manager.cc:1428] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:24:42.868419 6253 heartbeater.cc:503] Master 127.9.91.126:41525 was elected leader, sending a full tablet report...
I20260809 19:24:42.945611 6285 raft_consensus.cc:482] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:42.945705 6285 raft_consensus.cc:504] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.946031 6285 leader_election.cc:288] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4bc09165a174ff99bf83e63a7ebde5d (127.9.91.65:32947), adac7063c3994281b46c1de705704b10 (127.9.91.66:36429)
I20260809 19:24:42.949529 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.946272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:48775 (local address 127.9.91.66:36429)
0809 19:24:42.946313 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.946315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.946322 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:42.946349 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.946350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.946357 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.946403 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:42.946551 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.947288 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.948649 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.949204 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.949274 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.949283 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.949423 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.949463 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:24:42.949548 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.946254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:35377 (local address 127.9.91.65:32947)
0809 19:24:42.946293 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.946295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.946431 (+ 136us) server_negotiation.cc:407] Connection header received
0809 19:24:42.946457 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.946458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.946469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.946512 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:42.946651 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.947339 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.948649 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.949202 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.949274 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.949283 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.949430 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.949463 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:24:42.949566 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.946163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:36429 (local address 127.9.91.67:48775)
0809 19:24:42.946217 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.946230 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.946277 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.946420 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.946420 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.946422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:42.946488 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.946490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.947353 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.947354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.948560 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.948564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.949280 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.949281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.949356 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.949359 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.949413 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.949465 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":9}
I20260809 19:24:42.949581 6287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.946174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:32947 (local address 127.9.91.67:35377)
0809 19:24:42.946361 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.946369 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:42.946419 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:42.946528 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:42.946529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:42.946531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:42.946592 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.946594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.947400 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.947401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.948561 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:42.948564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.949279 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.949280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:42.949372 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.949374 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:42.949421 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:24:42.949463 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":68,"thread_start_us":87,"threads_started":1}
I20260809 19:24:42.949965 5980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d86c1dfd5d4fb9aed3d70e7265eba2" candidate_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adac7063c3994281b46c1de705704b10" is_pre_election: true
I20260809 19:24:42.950117 5980 raft_consensus.cc:2454] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45884d68cc84ff0a1cd5b53f89a47d2 in term 0.
I20260809 19:24:42.950170 5776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d86c1dfd5d4fb9aed3d70e7265eba2" candidate_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" is_pre_election: true
I20260809 19:24:42.950294 5776 raft_consensus.cc:2454] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45884d68cc84ff0a1cd5b53f89a47d2 in term 0.
I20260809 19:24:42.950345 6076 leader_election.cc:302] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adac7063c3994281b46c1de705704b10, b45884d68cc84ff0a1cd5b53f89a47d2; no voters:
I20260809 19:24:42.950457 6285 raft_consensus.cc:2781] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:24:42.950516 6285 raft_consensus.cc:482] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:24:42.950539 6285 raft_consensus.cc:3037] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:42.951215 6285 raft_consensus.cc:504] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.951329 6285 leader_election.cc:288] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [CANDIDATE]: Term 1 election: Requested vote from peers a4bc09165a174ff99bf83e63a7ebde5d (127.9.91.65:32947), adac7063c3994281b46c1de705704b10 (127.9.91.66:36429)
I20260809 19:24:42.951462 5776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d86c1dfd5d4fb9aed3d70e7265eba2" candidate_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4bc09165a174ff99bf83e63a7ebde5d"
I20260809 19:24:42.951531 5776 raft_consensus.cc:3037] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:42.952179 5776 raft_consensus.cc:2454] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45884d68cc84ff0a1cd5b53f89a47d2 in term 1.
I20260809 19:24:42.952278 5980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17d86c1dfd5d4fb9aed3d70e7265eba2" candidate_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adac7063c3994281b46c1de705704b10"
I20260809 19:24:42.952334 6079 leader_election.cc:302] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4bc09165a174ff99bf83e63a7ebde5d, b45884d68cc84ff0a1cd5b53f89a47d2; no voters:
I20260809 19:24:42.952351 5980 raft_consensus.cc:3037] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:42.952406 6285 raft_consensus.cc:2781] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:42.952512 6285 raft_consensus.cc:686] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 1 LEADER]: Becoming Leader. State: Replica: b45884d68cc84ff0a1cd5b53f89a47d2, State: Running, Role: LEADER
I20260809 19:24:42.952593 6285 consensus_queue.cc:237] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } }
I20260809 19:24:42.953159 5980 raft_consensus.cc:2454] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45884d68cc84ff0a1cd5b53f89a47d2 in term 1.
I20260809 19:24:42.953588 5558 catalog_manager.cc:5495] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b45884d68cc84ff0a1cd5b53f89a47d2 (127.9.91.67). New cstate: current_term: 1 leader_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45884d68cc84ff0a1cd5b53f89a47d2" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 32869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 } health_report { overall_health: UNKNOWN } } }
W20260809 19:24:42.958899 6053 tablet.cc:2369] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:24:42.969447 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.967594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.9.91.67:32869)
0809 19:24:42.967635 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:42.967636 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:42.967643 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:42.967668 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:42.967669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:42.967683 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:42.967735 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:42.967824 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.968236 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.968885 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:42.969061 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:42.969112 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:42.969213 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:42.969290 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:24:42.969352 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:42.969374 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 19:24:42.969381 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:24:42.970909 5776 raft_consensus.cc:1261] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P a4bc09165a174ff99bf83e63a7ebde5d [term 1 FOLLOWER]: Refusing update from remote peer b45884d68cc84ff0a1cd5b53f89a47d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:42.970932 5980 raft_consensus.cc:1261] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 1 FOLLOWER]: Refusing update from remote peer b45884d68cc84ff0a1cd5b53f89a47d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:42.971282 6290 consensus_queue.cc:1035] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adac7063c3994281b46c1de705704b10" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:42.971398 6285 consensus_queue.cc:1035] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4bc09165a174ff99bf83e63a7ebde5d" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 32947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:42.974083 6300 mvcc.cc:205] Tried to move back new op lower bound from 7316699066246852608 to 7316699066174476288. Current Snapshot: MvccSnapshot[applied={T|T < 7316699066246852608 or (T in {7316699066246852608})}]
I20260809 19:24:42.974203 6298 mvcc.cc:205] Tried to move back new op lower bound from 7316699066246852608 to 7316699066174476288. Current Snapshot: MvccSnapshot[applied={T|T < 7316699066246852608}]
I20260809 19:24:42.975718 5558 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:42.975795 5558 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:42.976456 5558 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea2dc4765f6341cda0bb1a5553c3e3d9: Sending DeleteTablet for 3 replicas of tablet 17d86c1dfd5d4fb9aed3d70e7265eba2
I20260809 19:24:42.976766 5947 tablet_service.cc:1514] Processing DeleteTablet for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:42 UTC) from {username='slave'} at 127.0.0.1:51856
I20260809 19:24:42.976768 5748 tablet_service.cc:1514] Processing DeleteTablet for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:42 UTC) from {username='slave'} at 127.0.0.1:59926
I20260809 19:24:42.976969 6311 tablet_replica.cc:330] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: stopping tablet replica
I20260809 19:24:42.976962 6152 tablet_service.cc:1514] Processing DeleteTablet for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:42 UTC) from {username='slave'} at 127.0.0.1:43256
I20260809 19:24:42.977036 6311 raft_consensus.cc:2229] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:42.977108 6311 raft_consensus.cc:2258] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:42.977167 6313 tablet_replica.cc:330] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: stopping tablet replica
I20260809 19:24:42.977331 6313 raft_consensus.cc:2229] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:24:42.977437 6313 raft_consensus.cc:2258] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:42.977429 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 5651
I20260809 19:24:42.978194 6313 ts_tablet_manager.cc:1905] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:42.980424 6313 ts_tablet_manager.cc:1918] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:42.980501 6313 log.cc:1196] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/17d86c1dfd5d4fb9aed3d70e7265eba2
I20260809 19:24:42.980767 6313 ts_tablet_manager.cc:1939] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P b45884d68cc84ff0a1cd5b53f89a47d2: Deleting consensus metadata
I20260809 19:24:42.981213 5542 catalog_manager.cc:4841] TS b45884d68cc84ff0a1cd5b53f89a47d2 (127.9.91.67:32869): tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 (table test-table [id=c2146fbc6cc346ba90fe4486c9f9cae1]) successfully deleted
I20260809 19:24:42.986965 6311 ts_tablet_manager.cc:1905] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:42.988086 6311 ts_tablet_manager.cc:1918] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:42.988152 6311 log.cc:1196] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/17d86c1dfd5d4fb9aed3d70e7265eba2
I20260809 19:24:42.988334 6311 ts_tablet_manager.cc:1939] T 17d86c1dfd5d4fb9aed3d70e7265eba2 P adac7063c3994281b46c1de705704b10: Deleting consensus metadata
I20260809 19:24:42.988662 5539 catalog_manager.cc:4841] TS adac7063c3994281b46c1de705704b10 (127.9.91.66:36429): tablet 17d86c1dfd5d4fb9aed3d70e7265eba2 (table test-table [id=c2146fbc6cc346ba90fe4486c9f9cae1]) successfully deleted
W20260809 19:24:42.990836 5540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:32947 (error 108)
I20260809 19:24:42.991294 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 5862
I20260809 19:24:42.991333 6268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:42.991220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:32947 (local address 127.0.0.1:59928)
0809 19:24:42.991268 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:42.991308 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 19:24:42.996289 5540 catalog_manager.cc:4584] TS a4bc09165a174ff99bf83e63a7ebde5d (127.9.91.65:32947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17d86c1dfd5d4fb9aed3d70e7265eba2: Network error: Client connection negotiation failed: client connection to 127.9.91.65:32947: connect: Connection refused (error 111)
I20260809 19:24:43.000264 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 6058
I20260809 19:24:43.009004 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 5511
2026-08-09T19:24:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (760 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:43 dist-test-slave-1vmg krb5kdc[6320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:43 dist-test-slave-1vmg krb5kdc[6320](info): set up 2 sockets
Aug 09 19:24:43 dist-test-slave-1vmg krb5kdc[6320](info): commencing operation
krb5kdc: starting...
W20260809 19:24:44.162863 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:24:44.269587 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:42923
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:42923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:44.347854 6342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:44.348038 6342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:44.348068 6342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:44.349161 6342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:44.349201 6342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:44.349220 6342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:44.349234 6342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:44.349251 6342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:44.350656 6342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:42923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:42923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:44.350876 6342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:44.351085 6342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:44.355489 6353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.355450 6349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.355470 6351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:44.355702 6342 server_base.cc:1034] running on GCE node
I20260809 19:24:44.355919 6342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:44.356181 6342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:44.357312 6342 hybrid_clock.cc:648] HybridClock initialized: now 1786303484357294 us; error 39 us; skew 500 ppm
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:44.367286 6342 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:24:44.367688 6342 webserver.cc:460] Webserver started at http://127.9.91.126:37587/ using document root <none> and password file <none>
I20260809 19:24:44.367875 6342 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:44.367921 6342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:44.368043 6342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:44.369012 6342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "c69228b10bf845e5bfaff33287015f1b"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "4375e5b22f29e259d05e1ce8cea1d64d"
server_key_iv: "51fb7999362c495be79a7ce30fbfef71"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.369333 6342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "c69228b10bf845e5bfaff33287015f1b"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "4375e5b22f29e259d05e1ce8cea1d64d"
server_key_iv: "51fb7999362c495be79a7ce30fbfef71"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.370553 6342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:24:44.371243 6361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.371395 6342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.371479 6342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "c69228b10bf845e5bfaff33287015f1b"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "4375e5b22f29e259d05e1ce8cea1d64d"
server_key_iv: "51fb7999362c495be79a7ce30fbfef71"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.371544 6342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:44.383821 6342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:44.385371 6342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:44.385492 6342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:44.389267 6342 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:42923
I20260809 19:24:44.389321 6433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:42923 every 8 connection(s)
I20260809 19:24:44.389605 6342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:44.390389 6434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:44.392627 6434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b: Bootstrap starting.
I20260809 19:24:44.393178 6434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:44.393409 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 6342
I20260809 19:24:44.393457 6434 log.cc:824] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:44.393512 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:44.393677 9581 external_mini_cluster.cc:1428] Setting key 695fcf980503c873fa7436c2e48bfc67
I20260809 19:24:44.394250 6434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b: No bootstrap required, opened a new log
I20260809 19:24:44.396355 6434 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69228b10bf845e5bfaff33287015f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42923 } }
I20260809 19:24:44.396477 6434 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:44.396508 6434 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69228b10bf845e5bfaff33287015f1b, State: Initialized, Role: FOLLOWER
I20260809 19:24:44.396653 6434 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69228b10bf845e5bfaff33287015f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42923 } }
I20260809 19:24:44.396739 6434 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:44.396770 6434 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:44.396813 6434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:44.397452 6434 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69228b10bf845e5bfaff33287015f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42923 } }
I20260809 19:24:44.397571 6434 leader_election.cc:302] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69228b10bf845e5bfaff33287015f1b; no voters:
I20260809 19:24:44.397734 6434 leader_election.cc:288] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:44.397787 6442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:44.397945 6442 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [term 1 LEADER]: Becoming Leader. State: Replica: c69228b10bf845e5bfaff33287015f1b, State: Running, Role: LEADER
I20260809 19:24:44.397967 6434 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:44.398041 6442 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69228b10bf845e5bfaff33287015f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42923 } }
I20260809 19:24:44.398666 6443 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c69228b10bf845e5bfaff33287015f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69228b10bf845e5bfaff33287015f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42923 } } }
I20260809 19:24:44.398751 6443 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:44.399070 6457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:44.399525 6457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:44.400314 6444 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69228b10bf845e5bfaff33287015f1b. Latest consensus state: current_term: 1 leader_uuid: "c69228b10bf845e5bfaff33287015f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69228b10bf845e5bfaff33287015f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 42923 } } }
I20260809 19:24:44.400715 6444 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:44.401202 6457 catalog_manager.cc:1347] Generated new cluster ID: 42a0be67ce9948a3918144958f52a7ca
I20260809 19:24:44.401252 6457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:44.404686 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.394150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.9.91.126:42923)
0809 19:24:44.394308 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.394317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.394352 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:24:44.394757 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.394763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.394809 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.394957 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:44.395639 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.396485 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.397585 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.398198 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.398355 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.401057 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:44.401061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:44.401061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:44.401073 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:44.403080 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.403455 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.403458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.403459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.403579 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.403762 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.403763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.403763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.404076 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:44.404236 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.404320 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.404473 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260809 19:24:44.408874 6457 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:44.409500 6457 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:44.420512 6457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b: Generated new TSK 0
I20260809 19:24:44.420674 6457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.91.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
Entry for principal kudu/127.9.91.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab.
I20260809 19:24:44.446498 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42923
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:44.525210 6480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:44.525394 6480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:44.525425 6480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:44.526568 6480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:44.526609 6480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:44.526643 6480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:44.528050 6480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.65.keytab
--principal=kudu/127.9.91.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:44.528264 6480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:44.528455 6480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:44.533099 6489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.533113 6486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.533170 6485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:44.533246 6480 server_base.cc:1034] running on GCE node
I20260809 19:24:44.533435 6480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:44.533661 6480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:44.534791 6480 hybrid_clock.cc:648] HybridClock initialized: now 1786303484534766 us; error 42 us; skew 500 ppm
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:44.537114 6480 init.cc:377] Logged in from keytab as kudu/127.9.91.65@KRBTEST.COM (short username kudu)
I20260809 19:24:44.537518 6480 webserver.cc:460] Webserver started at http://127.9.91.65:39029/ using document root <none> and password file <none>
I20260809 19:24:44.537712 6480 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:44.537758 6480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:44.537875 6480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:44.538810 6480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "2680e6781a4241c4b43afd8e087f7317"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "4a66a8d37d318c2049b535c9ab5e07a5"
server_key_iv: "abc8e3b8d524af9033561d75ea89203a"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.539178 6480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "2680e6781a4241c4b43afd8e087f7317"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "4a66a8d37d318c2049b535c9ab5e07a5"
server_key_iv: "abc8e3b8d524af9033561d75ea89203a"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.540364 6480 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:44.541016 6500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.541160 6480 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:44.541239 6480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "2680e6781a4241c4b43afd8e087f7317"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "4a66a8d37d318c2049b535c9ab5e07a5"
server_key_iv: "abc8e3b8d524af9033561d75ea89203a"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.541302 6480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:44.570407 6480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:44.572363 6480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:44.572487 6480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:44.572682 6480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:44.572947 6480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:44.572978 6480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.573004 6480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:44.573037 6480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.578435 6480 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:38025
I20260809 19:24:44.578500 6669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:38025 every 8 connection(s)
I20260809 19:24:44.578747 6480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:24:44.581207 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 6480
I20260809 19:24:44.581314 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:24:44.581499 9581 external_mini_cluster.cc:1428] Setting key 604c82f9571ba60a639f1fe381742d8f
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.65@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:44.587267 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.579328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:37131 (local address 127.9.91.126:42923)
0809 19:24:44.579367 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.579369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.579549 (+ 180us) server_negotiation.cc:407] Connection header received
0809 19:24:44.579951 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.579952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.579962 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.580005 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:44.580793 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.581265 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.582330 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.582550 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.582604 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.584636 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:44.584637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:44.584638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:44.584644 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:44.585867 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.586270 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.586271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.586272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.586338 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.586591 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.586592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.586592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.586713 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:44.586777 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.587077 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.587172 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:24:44.587445 6672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.579306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:42923 (local address 127.9.91.65:37131)
0809 19:24:44.579441 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.579472 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.579879 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.580065 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.580068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.580564 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:44.580730 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.580734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.581354 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.581355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.582252 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.582254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.582620 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.582621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.582725 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.583169 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:44.583176 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:44.584552 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:44.585961 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:44.585970 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:44.585972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:44.586193 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:44.586405 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:44.586406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:44.586407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:44.586525 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:44.586774 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:44.586782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:44.586910 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.587051 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.587198 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260809 19:24:44.587849 6670 heartbeater.cc:347] Connected to a master server at 127.9.91.126:42923
I20260809 19:24:44.587940 6670 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:44.588110 6670 heartbeater.cc:511] Master 127.9.91.126:42923 requested a full tablet report, sending...
I20260809 19:24:44.588634 6386 ts_manager.cc:194] Registered new tserver with Master: 2680e6781a4241c4b43afd8e087f7317 (127.9.91.65:38025)
I20260809 19:24:44.589221 6386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.65@KRBTEST.COM'} at 127.9.91.65:37131
WARNING: no policy specified for kudu/127.9.91.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
Entry for principal kudu/127.9.91.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab.
I20260809 19:24:44.605208 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42923
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:44.683079 6676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:44.683252 6676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:44.683274 6676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:44.684325 6676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:44.684357 6676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:44.684381 6676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:44.685736 6676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.66.keytab
--principal=kudu/127.9.91.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:44.685910 6676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:44.686071 6676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:44.690527 6686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.690527 6681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.690536 6682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:44.690752 6676 server_base.cc:1034] running on GCE node
I20260809 19:24:44.690943 6676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:44.691161 6676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:44.692304 6676 hybrid_clock.cc:648] HybridClock initialized: now 1786303484692294 us; error 28 us; skew 500 ppm
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:44.694540 6676 init.cc:377] Logged in from keytab as kudu/127.9.91.66@KRBTEST.COM (short username kudu)
I20260809 19:24:44.694952 6676 webserver.cc:460] Webserver started at http://127.9.91.66:35319/ using document root <none> and password file <none>
I20260809 19:24:44.695151 6676 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:44.695191 6676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:44.695276 6676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:44.696144 6676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "27c2431127bb4ae3b5c857ef6e715e64"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "6fb5c3e4877b128ad2bd03534d64bad3"
server_key_iv: "cda21f2e0cbd0afab3ab83d4bfe88f99"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.696416 6676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "27c2431127bb4ae3b5c857ef6e715e64"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "6fb5c3e4877b128ad2bd03534d64bad3"
server_key_iv: "cda21f2e0cbd0afab3ab83d4bfe88f99"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.697504 6676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:24:44.698103 6697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.698241 6676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:44.698316 6676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "27c2431127bb4ae3b5c857ef6e715e64"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "6fb5c3e4877b128ad2bd03534d64bad3"
server_key_iv: "cda21f2e0cbd0afab3ab83d4bfe88f99"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.698367 6676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:44.708118 6676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:44.709663 6676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:44.709761 6676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:44.709913 6676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:44.710131 6676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:44.710162 6676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.710180 6676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:44.710197 6676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.715438 6676 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:32953
I20260809 19:24:44.715504 6870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:32953 every 8 connection(s)
I20260809 19:24:44.715745 6676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:24:44.719357 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 6676
I20260809 19:24:44.719475 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:24:44.719655 9581 external_mini_cluster.cc:1428] Setting key 459fe9cead5138a0f8972979674e90f9
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.66@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:44.724088 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.716337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:50429 (local address 127.9.91.126:42923)
0809 19:24:44.716408 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.716411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.716568 (+ 157us) server_negotiation.cc:407] Connection header received
0809 19:24:44.716964 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.716966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.716976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.717021 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:44.717862 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.718360 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.719364 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.719653 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.719728 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.721592 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:44.721593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:44.721594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:44.721602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:44.722801 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.723236 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.723237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.723237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.723297 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.723547 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.723548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.723548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.723647 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:44.723693 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.723962 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.724025 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 19:24:44.724291 6873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.716306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:42923 (local address 127.9.91.66:50429)
0809 19:24:44.716469 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.716500 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.716895 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.717077 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.717080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.717586 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:44.717752 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.717756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.718439 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.718439 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.719288 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.719290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.719738 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.719739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.719858 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.720271 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:44.720281 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:44.721507 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:44.722873 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:44.722879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:44.722881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:44.723167 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:44.723356 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:44.723356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:44.723357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:44.723482 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:44.723695 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:44.723701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:44.723844 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.723949 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.724091 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260809 19:24:44.724725 6871 heartbeater.cc:347] Connected to a master server at 127.9.91.126:42923
I20260809 19:24:44.724804 6871 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:44.724958 6871 heartbeater.cc:511] Master 127.9.91.126:42923 requested a full tablet report, sending...
I20260809 19:24:44.725303 6386 ts_manager.cc:194] Registered new tserver with Master: 27c2431127bb4ae3b5c857ef6e715e64 (127.9.91.66:32953)
I20260809 19:24:44.725656 6386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.66@KRBTEST.COM'} at 127.9.91.66:50429
WARNING: no policy specified for kudu/127.9.91.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
Entry for principal kudu/127.9.91.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab.
I20260809 19:24:44.742852 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42923
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:44.821246 6878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:44.821434 6878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:44.821457 6878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:44.822525 6878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:44.822562 6878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:44.822588 6878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:44.824007 6878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.67.keytab
--principal=kudu/127.9.91.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:42923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:44.824191 6878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:44.824371 6878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:44.829180 6888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.829156 6889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:44.829172 6892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:44.829370 6878 server_base.cc:1034] running on GCE node
I20260809 19:24:44.829562 6878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:44.829815 6878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:44.830981 6878 hybrid_clock.cc:648] HybridClock initialized: now 1786303484830967 us; error 24 us; skew 500 ppm
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:44.833257 6878 init.cc:377] Logged in from keytab as kudu/127.9.91.67@KRBTEST.COM (short username kudu)
I20260809 19:24:44.833622 6878 webserver.cc:460] Webserver started at http://127.9.91.67:34015/ using document root <none> and password file <none>
I20260809 19:24:44.833802 6878 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:44.833840 6878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:44.833927 6878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:44.834798 6878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "1d3a3205152d4a7a8596ed75302baa4c"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "cc9467e7b76d526029be492c65e44902"
server_key_iv: "2c8df0b6ad887198a574427580936d37"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.835136 6878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "1d3a3205152d4a7a8596ed75302baa4c"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "cc9467e7b76d526029be492c65e44902"
server_key_iv: "2c8df0b6ad887198a574427580936d37"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.836294 6878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:44.836969 6899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.837131 6878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:44.837209 6878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "1d3a3205152d4a7a8596ed75302baa4c"
format_stamp: "Formatted at 2026-08-09 19:24:44 on dist-test-slave-1vmg"
server_key: "cc9467e7b76d526029be492c65e44902"
server_key_iv: "2c8df0b6ad887198a574427580936d37"
server_key_version: "encryptionkey@0"
I20260809 19:24:44.837267 6878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:44.859790 6878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:44.861726 6878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:44.861855 6878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:44.862043 6878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:44.862313 6878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:44.862346 6878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.862372 6878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:44.862389 6878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:44.867930 6878 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:40035
I20260809 19:24:44.867991 7069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:40035 every 8 connection(s)
I20260809 19:24:44.868307 6878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
Aug 09 19:24:44 dist-test-slave-1vmg krb5kdc[6320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303484, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.67@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:24:44.876451 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.868902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:33867 (local address 127.9.91.126:42923)
0809 19:24:44.868949 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.868951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.869138 (+ 187us) server_negotiation.cc:407] Connection header received
0809 19:24:44.869527 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.869528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.869544 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.869592 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:44.870412 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.870893 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.871860 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.872057 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.872108 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.873911 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:44.873912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:44.873912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:44.873921 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:44.875247 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.875597 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.875598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.875598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.875659 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.875867 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.875868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.875868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.875977 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:44.876032 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.876281 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.876361 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 19:24:44.876588 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.868859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:42923 (local address 127.9.91.67:33867)
0809 19:24:44.869021 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.869059 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.869455 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.869655 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.869658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.870169 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:44.870346 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.870351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.871023 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.871024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.871787 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.871789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.872116 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.872117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.872210 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.872566 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:24:44.872575 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:44.873822 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:44.875323 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:44.875330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:44.875332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:44.875530 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:44.875711 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:24:44.875712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:24:44.875713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:24:44.875812 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:24:44.876031 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:44.876046 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:44.876168 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.876251 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.876401 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260809 19:24:44.876987 7070 heartbeater.cc:347] Connected to a master server at 127.9.91.126:42923
I20260809 19:24:44.877079 7070 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:44.877239 7070 heartbeater.cc:511] Master 127.9.91.126:42923 requested a full tablet report, sending...
I20260809 19:24:44.877242 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 6878
I20260809 19:24:44.877347 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:24:44.877528 9581 external_mini_cluster.cc:1428] Setting key e6be4dcd9d47784a039463064fce6328
I20260809 19:24:44.877630 6386 ts_manager.cc:194] Registered new tserver with Master: 1d3a3205152d4a7a8596ed75302baa4c (127.9.91.67:40035)
I20260809 19:24:44.878049 6386 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.91.67@KRBTEST.COM'} at 127.9.91.67:33867
I20260809 19:24:44.878400 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:24:44.884493 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.879695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.9.91.126:42923)
0809 19:24:44.879757 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.879758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.879868 (+ 110us) server_negotiation.cc:407] Connection header received
0809 19:24:44.879918 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.879919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.879927 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.879961 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:44.880439 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.880862 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.881491 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.881656 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.881694 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.882597 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:44.882598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:44.882599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:44.882604 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:44.883739 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.883993 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.883993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.883993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.884033 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:44.884149 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:44.884150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:44.884150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:44.884260 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:44.884325 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.884391 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.884441 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:24:44.885825 6385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:24:44.887339 6385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:24:44.893160 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.889381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37716 (local address 127.9.91.66:32953)
0809 19:24:44.889827 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.889830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.889837 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:44.889865 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.889866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.889878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.890039 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:44.890233 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.891198 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.892213 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.892781 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.892851 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.892900 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.893044 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.893084 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":78,"threads_started":1}
I20260809 19:24:44.893159 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.889192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38025 (local address 127.0.0.1:41828)
0809 19:24:44.889389 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.889426 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.889486 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.889730 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.889732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.889734 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:44.889782 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.889783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.890962 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.890963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.892172 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.892175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.892960 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.892961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.893013 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.893016 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.893058 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.893089 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":54,"threads_started":1}
I20260809 19:24:44.893400 7085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.889354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.9.91.65:38025)
0809 19:24:44.889591 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.889595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.889605 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:24:44.889637 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.889639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.889650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.889713 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:44.889851 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.890832 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.892263 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.892817 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.892874 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.892928 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.893277 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.893324 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":85,"threads_started":1}
I20260809 19:24:44.893160 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.889195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:32953 (local address 127.0.0.1:37716)
0809 19:24:44.889529 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.889538 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.889577 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.890105 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.890106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.890108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:44.890154 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.890155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.891282 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.891284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.892130 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.892133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.892866 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.892867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.892942 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.892955 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.893018 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.893076 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":27,"thread_start_us":44,"threads_started":1}
I20260809 19:24:44.893586 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.889375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.9.91.67:40035)
0809 19:24:44.890166 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.890170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.890179 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:44.890209 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.890210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.890223 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.890291 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:44.890452 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.891225 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.892496 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.893123 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.893187 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.893239 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.893473 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.893520 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":80,"threads_started":1}
I20260809 19:24:44.893620 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.889195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40035 (local address 127.0.0.1:59222)
0809 19:24:44.889638 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.889644 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.889668 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.890353 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.890354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.890356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:44.890389 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.890390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.891335 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.891336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.892419 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.892422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.893279 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.893280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.893342 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.893347 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.893391 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.893430 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":7,"thread_start_us":70,"threads_started":1}
I20260809 19:24:44.894042 6567 tablet_service.cc:1467] Processing CreateTablet for tablet b0991c88738345d19ec6efbeaaff611a (DEFAULT_TABLE table=test-table [id=5a082e7830fa40e0b3e661f397ff76b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:44.894400 6567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0991c88738345d19ec6efbeaaff611a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:44.894338 6974 tablet_service.cc:1467] Processing CreateTablet for tablet b0991c88738345d19ec6efbeaaff611a (DEFAULT_TABLE table=test-table [id=5a082e7830fa40e0b3e661f397ff76b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:44.894340 6775 tablet_service.cc:1467] Processing CreateTablet for tablet b0991c88738345d19ec6efbeaaff611a (DEFAULT_TABLE table=test-table [id=5a082e7830fa40e0b3e661f397ff76b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:44.894629 6775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0991c88738345d19ec6efbeaaff611a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:44.894629 6974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0991c88738345d19ec6efbeaaff611a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:44.896953 7090 tablet_bootstrap.cc:492] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317: Bootstrap starting.
I20260809 19:24:44.897665 7091 tablet_bootstrap.cc:492] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Bootstrap starting.
I20260809 19:24:44.897910 7090 tablet_bootstrap.cc:654] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:44.898322 7091 tablet_bootstrap.cc:654] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:44.898458 7090 log.cc:824] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:44.898674 7092 tablet_bootstrap.cc:492] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Bootstrap starting.
I20260809 19:24:44.899248 7092 tablet_bootstrap.cc:654] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:44.899338 7090 tablet_bootstrap.cc:492] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317: No bootstrap required, opened a new log
I20260809 19:24:44.899408 7090 ts_tablet_manager.cc:1397] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 19:24:44.899827 7092 log.cc:824] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:44.900411 7091 log.cc:824] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:44.900998 7092 tablet_bootstrap.cc:492] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: No bootstrap required, opened a new log
I20260809 19:24:44.901081 7092 ts_tablet_manager.cc:1397] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:44.901226 7091 tablet_bootstrap.cc:492] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: No bootstrap required, opened a new log
I20260809 19:24:44.901286 7091 ts_tablet_manager.cc:1397] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 19:24:44.901392 7090 raft_consensus.cc:348] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.901583 7090 raft_consensus.cc:374] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:44.901628 7090 raft_consensus.cc:729] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2680e6781a4241c4b43afd8e087f7317, State: Initialized, Role: FOLLOWER
I20260809 19:24:44.901736 7090 consensus_queue.cc:260] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.902034 7090 ts_tablet_manager.cc:1428] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:24:44.902177 6670 heartbeater.cc:503] Master 127.9.91.126:42923 was elected leader, sending a full tablet report...
I20260809 19:24:44.902766 7092 raft_consensus.cc:348] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.902927 7092 raft_consensus.cc:374] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:44.902976 7092 raft_consensus.cc:729] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d3a3205152d4a7a8596ed75302baa4c, State: Initialized, Role: FOLLOWER
I20260809 19:24:44.903107 7092 consensus_queue.cc:260] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.903098 7091 raft_consensus.cc:348] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.903254 7091 raft_consensus.cc:374] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:44.903296 7091 raft_consensus.cc:729] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27c2431127bb4ae3b5c857ef6e715e64, State: Initialized, Role: FOLLOWER
I20260809 19:24:44.903367 7092 ts_tablet_manager.cc:1428] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:24:44.903381 7070 heartbeater.cc:503] Master 127.9.91.126:42923 was elected leader, sending a full tablet report...
I20260809 19:24:44.903451 7091 consensus_queue.cc:260] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.903712 7091 ts_tablet_manager.cc:1428] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 19:24:44.903726 6871 heartbeater.cc:503] Master 127.9.91.126:42923 was elected leader, sending a full tablet report...
W20260809 19:24:44.966264 6872 tablet.cc:2369] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:24:44.991871 7103 raft_consensus.cc:482] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:44.991968 7103 raft_consensus.cc:504] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.992353 7103 leader_election.cc:288] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2680e6781a4241c4b43afd8e087f7317 (127.9.91.65:38025), 1d3a3205152d4a7a8596ed75302baa4c (127.9.91.67:40035)
I20260809 19:24:44.996052 7085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.992585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:44531 (local address 127.9.91.65:38025)
0809 19:24:44.992625 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.992627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.992782 (+ 155us) server_negotiation.cc:407] Connection header received
0809 19:24:44.992810 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.992811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.992820 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.992859 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:44.992986 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.993664 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.995127 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.995698 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.995767 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.995777 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.995931 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.995970 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:24:44.996042 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.992621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:53823 (local address 127.9.91.67:40035)
0809 19:24:44.992665 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:44.992667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:44.992675 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:44.992700 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:44.992701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:44.992712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:44.992762 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:44.992918 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.993656 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.995128 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.995711 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.995766 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.995776 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:44.995931 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 19:24:44.995970 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:24:44.996058 6873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.992488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40035 (local address 127.9.91.66:53823)
0809 19:24:44.992545 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.992558 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.992608 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.992777 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.992778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.992779 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:44.992850 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.992852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.993731 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.993731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.995053 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.995057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.995780 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.995781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.995865 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.995871 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.995920 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.995966 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":51}
I20260809 19:24:44.996060 7104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:44.992488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38025 (local address 127.9.91.66:44531)
0809 19:24:44.992708 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:44.992717 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:44.992769 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:44.992862 (+ 93us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:44.992863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:44.992874 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:44.992926 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.992927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.993731 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.993731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.995053 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:44.995057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:44.995780 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:44.995781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:44.995865 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:44.995871 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 19:24:44.995920 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 19:24:44.995966 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":165,"thread_start_us":92,"threads_started":1}
I20260809 19:24:44.996420 6998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0991c88738345d19ec6efbeaaff611a" candidate_uuid: "27c2431127bb4ae3b5c857ef6e715e64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3a3205152d4a7a8596ed75302baa4c" is_pre_election: true
I20260809 19:24:44.996573 6998 raft_consensus.cc:2454] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c2431127bb4ae3b5c857ef6e715e64 in term 0.
I20260809 19:24:44.996647 6597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0991c88738345d19ec6efbeaaff611a" candidate_uuid: "27c2431127bb4ae3b5c857ef6e715e64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2680e6781a4241c4b43afd8e087f7317" is_pre_election: true
I20260809 19:24:44.996788 6597 raft_consensus.cc:2454] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27c2431127bb4ae3b5c857ef6e715e64 in term 0.
I20260809 19:24:44.996791 6701 leader_election.cc:302] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d3a3205152d4a7a8596ed75302baa4c, 27c2431127bb4ae3b5c857ef6e715e64; no voters:
I20260809 19:24:44.996904 7103 raft_consensus.cc:2781] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:24:44.996969 7103 raft_consensus.cc:482] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:24:44.997002 7103 raft_consensus.cc:3037] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:44.997735 7103 raft_consensus.cc:504] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:44.997851 7103 leader_election.cc:288] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [CANDIDATE]: Term 1 election: Requested vote from peers 2680e6781a4241c4b43afd8e087f7317 (127.9.91.65:38025), 1d3a3205152d4a7a8596ed75302baa4c (127.9.91.67:40035)
I20260809 19:24:44.997994 6998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0991c88738345d19ec6efbeaaff611a" candidate_uuid: "27c2431127bb4ae3b5c857ef6e715e64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d3a3205152d4a7a8596ed75302baa4c"
I20260809 19:24:44.998003 6597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0991c88738345d19ec6efbeaaff611a" candidate_uuid: "27c2431127bb4ae3b5c857ef6e715e64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2680e6781a4241c4b43afd8e087f7317"
I20260809 19:24:44.998077 6998 raft_consensus.cc:3037] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:44.998077 6597 raft_consensus.cc:3037] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:44.998997 6998 raft_consensus.cc:2454] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c2431127bb4ae3b5c857ef6e715e64 in term 1.
I20260809 19:24:44.998997 6597 raft_consensus.cc:2454] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27c2431127bb4ae3b5c857ef6e715e64 in term 1.
I20260809 19:24:44.999151 6701 leader_election.cc:302] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d3a3205152d4a7a8596ed75302baa4c, 27c2431127bb4ae3b5c857ef6e715e64; no voters:
I20260809 19:24:44.999248 7103 raft_consensus.cc:2781] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:44.999405 7103 raft_consensus.cc:686] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 1 LEADER]: Becoming Leader. State: Replica: 27c2431127bb4ae3b5c857ef6e715e64, State: Running, Role: LEADER
I20260809 19:24:44.999495 7103 consensus_queue.cc:237] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } }
I20260809 19:24:45.000242 6386 catalog_manager.cc:5495] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27c2431127bb4ae3b5c857ef6e715e64 (127.9.91.66). New cstate: current_term: 1 leader_uuid: "27c2431127bb4ae3b5c857ef6e715e64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27c2431127bb4ae3b5c857ef6e715e64" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 32953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 } health_report { overall_health: UNKNOWN } } }
I20260809 19:24:45.006165 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.004086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.9.91.66:32953)
0809 19:24:45.004133 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.004135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.004142 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:24:45.004170 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.004171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.004188 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.004247 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:45.004363 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.004796 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.005499 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.005698 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.005749 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.005889 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:45.005982 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:24:45.006050 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.006086 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.006103 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:24:45.007655 6597 raft_consensus.cc:1261] T b0991c88738345d19ec6efbeaaff611a P 2680e6781a4241c4b43afd8e087f7317 [term 1 FOLLOWER]: Refusing update from remote peer 27c2431127bb4ae3b5c857ef6e715e64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:45.007655 6998 raft_consensus.cc:1261] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 1 FOLLOWER]: Refusing update from remote peer 27c2431127bb4ae3b5c857ef6e715e64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:45.008001 7105 consensus_queue.cc:1035] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d3a3205152d4a7a8596ed75302baa4c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:45.008122 7103 consensus_queue.cc:1035] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2680e6781a4241c4b43afd8e087f7317" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 38025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:45.010128 7109 mvcc.cc:205] Tried to move back new op lower bound from 7316699074589048832 to 7316699074558459904. Current Snapshot: MvccSnapshot[applied={T|T < 7316699074589048832}]
I20260809 19:24:45.010510 7111 mvcc.cc:205] Tried to move back new op lower bound from 7316699074589048832 to 7316699074558459904. Current Snapshot: MvccSnapshot[applied={T|T < 7316699074589048832}]
I20260809 19:24:45.012349 6385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:45.012431 6385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:45.013078 6385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c69228b10bf845e5bfaff33287015f1b: Sending DeleteTablet for 3 replicas of tablet b0991c88738345d19ec6efbeaaff611a
I20260809 19:24:45.013372 6775 tablet_service.cc:1514] Processing DeleteTablet for tablet b0991c88738345d19ec6efbeaaff611a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:45 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:37716
I20260809 19:24:45.013376 6567 tablet_service.cc:1514] Processing DeleteTablet for tablet b0991c88738345d19ec6efbeaaff611a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:45 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:41828
I20260809 19:24:45.013402 6974 tablet_service.cc:1514] Processing DeleteTablet for tablet b0991c88738345d19ec6efbeaaff611a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:45 UTC) from {username='kudu', principal='kudu/127.9.91.126@KRBTEST.COM'} at 127.0.0.1:59222
I20260809 19:24:45.013775 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 6480
I20260809 19:24:45.013675 7124 tablet_replica.cc:330] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: stopping tablet replica
I20260809 19:24:45.038797 7125 tablet_replica.cc:330] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: stopping tablet replica
I20260809 19:24:45.038836 7124 raft_consensus.cc:2229] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 1 LEADER]: Raft consensus shutting down.
I20260809 19:24:45.038900 7125 raft_consensus.cc:2229] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:45.039052 7125 raft_consensus.cc:2258] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:45.039139 7124 raft_consensus.cc:2258] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:45.039609 7125 ts_tablet_manager.cc:1905] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:45.039841 7124 ts_tablet_manager.cc:1905] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:45.041388 7124 ts_tablet_manager.cc:1918] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:45.041388 7125 ts_tablet_manager.cc:1918] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:45.041461 7125 log.cc:1196] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/b0991c88738345d19ec6efbeaaff611a
I20260809 19:24:45.041574 7124 log.cc:1196] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/b0991c88738345d19ec6efbeaaff611a
I20260809 19:24:45.041738 7125 ts_tablet_manager.cc:1939] T b0991c88738345d19ec6efbeaaff611a P 1d3a3205152d4a7a8596ed75302baa4c: Deleting consensus metadata
I20260809 19:24:45.041932 7124 ts_tablet_manager.cc:1939] T b0991c88738345d19ec6efbeaaff611a P 27c2431127bb4ae3b5c857ef6e715e64: Deleting consensus metadata
I20260809 19:24:45.042152 6364 catalog_manager.cc:4841] TS 1d3a3205152d4a7a8596ed75302baa4c (127.9.91.67:40035): tablet b0991c88738345d19ec6efbeaaff611a (table test-table [id=5a082e7830fa40e0b3e661f397ff76b0]) successfully deleted
I20260809 19:24:45.042378 6366 catalog_manager.cc:4841] TS 27c2431127bb4ae3b5c857ef6e715e64 (127.9.91.66:32953): tablet b0991c88738345d19ec6efbeaaff611a (table test-table [id=5a082e7830fa40e0b3e661f397ff76b0]) successfully deleted
W20260809 19:24:45.044482 6363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:38025 (error 108)
I20260809 19:24:45.044909 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.044801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:38025 (local address 127.0.0.1:41836)
0809 19:24:45.044845 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.044881 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:38025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:24:45.045040 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 6676
W20260809 19:24:45.045047 6363 catalog_manager.cc:4584] TS 2680e6781a4241c4b43afd8e087f7317 (127.9.91.65:38025): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0991c88738345d19ec6efbeaaff611a: Network error: Client connection negotiation failed: client connection to 127.9.91.65:38025: connect: Connection refused (error 111)
I20260809 19:24:45.060686 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 6878
I20260809 19:24:45.070742 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 6342
2026-08-09T19:24:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2064 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T19:24:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:45Z Disabled control of system clock
I20260809 19:24:45.102936 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38815
--webserver_interface=127.9.91.126
--webserver_port=0
--builtin_ntp_servers=127.9.91.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:38815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:45.180542 7134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:45.180727 7134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:45.180745 7134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:45.181833 7134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:45.181872 7134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:45.181885 7134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:45.181896 7134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:45.181921 7134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:45.183229 7134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:38815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:45.183437 7134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:45.183696 7134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:45.188002 7145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:45.187989 7149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:45.188042 7146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:45.188089 7134 server_base.cc:1034] running on GCE node
I20260809 19:24:45.188428 7134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:45.188668 7134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:45.189826 7134 hybrid_clock.cc:648] HybridClock initialized: now 1786303485189803 us; error 29 us; skew 500 ppm
I20260809 19:24:45.190996 7134 webserver.cc:460] Webserver started at http://127.9.91.126:34781/ using document root <none> and password file <none>
I20260809 19:24:45.191195 7134 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:45.191237 7134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:45.191354 7134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:45.192277 7134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "e488989f07f944f4866f8758570c6758"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "1cd0642cff2374760fec8c27f62f9a87"
server_key_iv: "c2a7ca7eff0b063f0b252d5bdd5dbe5f"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.192587 7134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "e488989f07f944f4866f8758570c6758"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "1cd0642cff2374760fec8c27f62f9a87"
server_key_iv: "c2a7ca7eff0b063f0b252d5bdd5dbe5f"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.193706 7134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 19:24:45.194310 7158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.194442 7134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:45.194521 7134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "e488989f07f944f4866f8758570c6758"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "1cd0642cff2374760fec8c27f62f9a87"
server_key_iv: "c2a7ca7eff0b063f0b252d5bdd5dbe5f"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.194588 7134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:45.232959 7134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:45.233210 7134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:45.233331 7134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:45.237016 7134 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:38815
I20260809 19:24:45.237054 7252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:38815 every 8 connection(s)
I20260809 19:24:45.237337 7134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:45.238193 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:45.240285 7253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758: Bootstrap starting.
I20260809 19:24:45.240799 7253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:45.241027 7253 log.cc:824] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:45.241601 7253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758: No bootstrap required, opened a new log
I20260809 19:24:45.243100 7253 raft_consensus.cc:348] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e488989f07f944f4866f8758570c6758" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38815 } }
I20260809 19:24:45.243222 7253 raft_consensus.cc:374] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:45.243243 7253 raft_consensus.cc:729] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e488989f07f944f4866f8758570c6758, State: Initialized, Role: FOLLOWER
I20260809 19:24:45.243340 7253 consensus_queue.cc:260] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e488989f07f944f4866f8758570c6758" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38815 } }
I20260809 19:24:45.243388 7253 raft_consensus.cc:388] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:45.243407 7253 raft_consensus.cc:482] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:45.243434 7253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:45.243868 7253 raft_consensus.cc:504] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e488989f07f944f4866f8758570c6758" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38815 } }
I20260809 19:24:45.243950 7253 leader_election.cc:302] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e488989f07f944f4866f8758570c6758; no voters:
I20260809 19:24:45.244069 7253 leader_election.cc:288] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:45.244123 7256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:45.244280 7256 raft_consensus.cc:686] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [term 1 LEADER]: Becoming Leader. State: Replica: e488989f07f944f4866f8758570c6758, State: Running, Role: LEADER
I20260809 19:24:45.244287 7253 sys_catalog.cc:564] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:45.244383 7256 consensus_queue.cc:237] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e488989f07f944f4866f8758570c6758" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38815 } }
I20260809 19:24:45.244839 7259 sys_catalog.cc:455] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e488989f07f944f4866f8758570c6758. Latest consensus state: current_term: 1 leader_uuid: "e488989f07f944f4866f8758570c6758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e488989f07f944f4866f8758570c6758" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38815 } } }
I20260809 19:24:45.244920 7259 sys_catalog.cc:458] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:45.245146 7271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:45.245323 7258 sys_catalog.cc:455] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e488989f07f944f4866f8758570c6758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e488989f07f944f4866f8758570c6758" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38815 } } }
I20260809 19:24:45.245389 7258 sys_catalog.cc:458] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:45.245813 7271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:45.247184 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 7134
I20260809 19:24:45.247231 7271 catalog_manager.cc:1347] Generated new cluster ID: 6ff69c14a4cd4867ba12a1a6dc4aeb1b
I20260809 19:24:45.247280 7271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:45.247296 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:45.247501 9581 external_mini_cluster.cc:1428] Setting key 36fa4e06d5095e5c25c6a60ddc05b0ad
I20260809 19:24:45.254881 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.248016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.9.91.126:38815)
0809 19:24:45.251033 (+ 3017us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.251048 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.251070 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:24:45.251153 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.251158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.251206 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.251315 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:45.251524 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.252400 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.253150 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.253599 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.253707 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.254300 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:45.254304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:45.254304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:45.254453 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:45.254484 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:45.254494 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:45.254548 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.254627 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.254699 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2956,"thread_start_us":74,"threads_started":1}
I20260809 19:24:45.261274 7271 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:45.261715 7271 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:45.265419 7271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758: Generated new TSK 0
I20260809 19:24:45.265559 7271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:45.267107 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.65:0
--local_ip_for_outbound_sockets=127.9.91.65
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38815
--builtin_ntp_servers=127.9.91.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:45.375247 7285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:45.375423 7285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:45.375443 7285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:45.376559 7285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:45.376596 7285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:45.376619 7285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.65
I20260809 19:24:45.377912 7285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.91.65
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.65
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:45.378126 7285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:45.378389 7285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:45.382781 7291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:45.382848 7292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:45.383054 7296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:45.383096 7285 server_base.cc:1034] running on GCE node
I20260809 19:24:45.383283 7285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:45.383503 7285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:45.384641 7285 hybrid_clock.cc:648] HybridClock initialized: now 1786303485384626 us; error 32 us; skew 500 ppm
I20260809 19:24:45.385850 7285 webserver.cc:460] Webserver started at http://127.9.91.65:33567/ using document root <none> and password file <none>
I20260809 19:24:45.386067 7285 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:45.386114 7285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:45.386246 7285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:45.387243 7285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data/instance:
uuid: "597a3232f5a54e76bb938321339c32a9"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "2d67f9843dee5ec9b6db0b01c19e5c95"
server_key_iv: "b3e2cfe38067f3819a38fda7a671fee5"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.387553 7285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance:
uuid: "597a3232f5a54e76bb938321339c32a9"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "2d67f9843dee5ec9b6db0b01c19e5c95"
server_key_iv: "b3e2cfe38067f3819a38fda7a671fee5"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.388721 7285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:24:45.389403 7304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.389568 7285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:45.389649 7285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/wal
uuid: "597a3232f5a54e76bb938321339c32a9"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "2d67f9843dee5ec9b6db0b01c19e5c95"
server_key_iv: "b3e2cfe38067f3819a38fda7a671fee5"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.389709 7285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:45.403285 7285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:45.403522 7285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:45.403640 7285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:45.403822 7285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:45.404117 7285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:45.404157 7285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.404183 7285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:45.404209 7285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.409718 7285 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.65:40741
I20260809 19:24:45.409754 7485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.65:40741 every 8 connection(s)
I20260809 19:24:45.410046 7285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/data/info.pb
I20260809 19:24:45.414589 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.410708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:53809 (local address 127.9.91.126:38815)
0809 19:24:45.410773 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.410775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.411006 (+ 231us) server_negotiation.cc:407] Connection header received
0809 19:24:45.411480 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.411482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.411493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.411532 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:45.412153 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.412626 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.413521 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.413698 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.413738 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.414393 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:45.414395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:45.414395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:45.414399 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:45.414413 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:45.414417 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:45.414454 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.414507 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.414533 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:24:45.414747 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.410691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38815 (local address 127.9.91.65:53809)
0809 19:24:45.410870 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.410902 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.411390 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.411586 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.411589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.411597 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:45.412084 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.412089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.412692 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.412693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.413459 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.413462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.413750 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.413751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.413852 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.414277 (+ 425us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:45.414277 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:45.414303 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:45.414323 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:45.414456 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:45.414458 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:45.414460 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.414503 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.414575 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260809 19:24:45.415220 7486 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38815
I20260809 19:24:45.415299 7486 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:45.415441 7486 heartbeater.cc:511] Master 127.9.91.126:38815 requested a full tablet report, sending...
I20260809 19:24:45.415851 7190 ts_manager.cc:194] Registered new tserver with Master: 597a3232f5a54e76bb938321339c32a9 (127.9.91.65:40741)
I20260809 19:24:45.416433 7190 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.65:53809
I20260809 19:24:45.416718 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 7285
I20260809 19:24:45.416823 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-0/wal/instance
I20260809 19:24:45.417062 9581 external_mini_cluster.cc:1428] Setting key 074dd3ae17c474e39cf1212bebb476bf
I20260809 19:24:45.417892 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.66:0
--local_ip_for_outbound_sockets=127.9.91.66
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38815
--builtin_ntp_servers=127.9.91.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:45.497758 7493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:45.497951 7493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:45.497972 7493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:45.499101 7493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:45.499136 7493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:45.499159 7493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.66
I20260809 19:24:45.500434 7493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.91.66
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.66
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:45.500618 7493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:45.500846 7493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:45.505256 7502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:45.505336 7500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:45.505429 7493 server_base.cc:1034] running on GCE node
W20260809 19:24:45.505505 7504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:45.505715 7493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:45.505924 7493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:45.507057 7493 hybrid_clock.cc:648] HybridClock initialized: now 1786303485507042 us; error 26 us; skew 500 ppm
I20260809 19:24:45.508157 7493 webserver.cc:460] Webserver started at http://127.9.91.66:44479/ using document root <none> and password file <none>
I20260809 19:24:45.508378 7493 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:45.508424 7493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:45.508560 7493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:45.509490 7493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data/instance:
uuid: "fbc6860699d441beaa573f32d2a7d697"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "3caf8da8465bc5f96115276747bc88bb"
server_key_iv: "e80e7c9df4722e110b939cdc6be93788"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.509814 7493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance:
uuid: "fbc6860699d441beaa573f32d2a7d697"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "3caf8da8465bc5f96115276747bc88bb"
server_key_iv: "e80e7c9df4722e110b939cdc6be93788"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.511019 7493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:24:45.511658 7511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.511811 7493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.511890 7493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal
uuid: "fbc6860699d441beaa573f32d2a7d697"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "3caf8da8465bc5f96115276747bc88bb"
server_key_iv: "e80e7c9df4722e110b939cdc6be93788"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.511955 7493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:45.527882 7493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:45.528111 7493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:45.528226 7493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:45.528407 7493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:45.528688 7493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:45.528721 7493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.528746 7493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:45.528764 7493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.533985 7493 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.66:33507
I20260809 19:24:45.534051 7693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.66:33507 every 8 connection(s)
I20260809 19:24:45.534294 7493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/data/info.pb
I20260809 19:24:45.538902 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.534908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.66:45819 (local address 127.9.91.126:38815)
0809 19:24:45.534984 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.534986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.535194 (+ 208us) server_negotiation.cc:407] Connection header received
0809 19:24:45.535641 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.535643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.535655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.535702 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:45.536322 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.536817 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.537738 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.537926 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.537980 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.538641 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:45.538643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:45.538643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:45.538647 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:45.538663 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:45.538667 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:45.538711 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.538791 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.538825 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 19:24:45.539078 7701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.534882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38815 (local address 127.9.91.66:45819)
0809 19:24:45.535086 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.535115 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.535532 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.535775 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.535779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.535788 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:45.536231 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.536235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.536902 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.536904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.537649 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.537652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.537997 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.537999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.538119 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.538509 (+ 390us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:45.538509 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:45.538534 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:45.538561 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:45.538717 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:45.538722 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:45.538725 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.538786 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.538878 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":106,"threads_started":1}
I20260809 19:24:45.539494 7695 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38815
I20260809 19:24:45.539587 7695 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:45.539762 7695 heartbeater.cc:511] Master 127.9.91.126:38815 requested a full tablet report, sending...
I20260809 19:24:45.540072 7190 ts_manager.cc:194] Registered new tserver with Master: fbc6860699d441beaa573f32d2a7d697 (127.9.91.66:33507)
I20260809 19:24:45.540392 7190 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.66:45819
I20260809 19:24:45.543099 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 7493
I20260809 19:24:45.543203 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal/instance
I20260809 19:24:45.543367 9581 external_mini_cluster.cc:1428] Setting key 1685a7826c71efd34b3f0d4d6d96a291
I20260809 19:24:45.544134 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.91.67:0
--local_ip_for_outbound_sockets=127.9.91.67
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38815
--builtin_ntp_servers=127.9.91.84:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 19:24:45.623998 7702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:45.624171 7702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:45.624191 7702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:45.625233 7702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:45.625267 7702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:45.625289 7702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.91.67
I20260809 19:24:45.626520 7702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.91.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.91.67
--webserver_port=0
--tserver_master_addrs=127.9.91.126:38815
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.91.67
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:45.626679 7702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:45.626904 7702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:45.631409 7715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:45.631404 7710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:45.631487 7713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:45.631759 7702 server_base.cc:1034] running on GCE node
I20260809 19:24:45.631950 7702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:45.632143 7702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:45.633270 7702 hybrid_clock.cc:648] HybridClock initialized: now 1786303485633257 us; error 29 us; skew 500 ppm
I20260809 19:24:45.634424 7702 webserver.cc:460] Webserver started at http://127.9.91.67:35681/ using document root <none> and password file <none>
I20260809 19:24:45.634626 7702 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:45.634671 7702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:45.634790 7702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:45.635802 7702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data/instance:
uuid: "1adbaac82137497da7d374229f08d75c"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "28dc443e170517bc2cab1ae418f229c6"
server_key_iv: "d5b8b350d73aa2c3d6298e1b751934e4"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.636117 7702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance:
uuid: "1adbaac82137497da7d374229f08d75c"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "28dc443e170517bc2cab1ae418f229c6"
server_key_iv: "d5b8b350d73aa2c3d6298e1b751934e4"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.637279 7702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:24:45.637893 7722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.638044 7702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:45.638120 7702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal
uuid: "1adbaac82137497da7d374229f08d75c"
format_stamp: "Formatted at 2026-08-09 19:24:45 on dist-test-slave-1vmg"
server_key: "28dc443e170517bc2cab1ae418f229c6"
server_key_iv: "d5b8b350d73aa2c3d6298e1b751934e4"
server_key_version: "encryptionkey@0"
I20260809 19:24:45.638183 7702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:45.662601 7702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:45.662840 7702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:45.662978 7702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:45.663190 7702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 19:24:45.663475 7702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 19:24:45.663507 7702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.663538 7702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 19:24:45.663561 7702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:45.668937 7702 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.67:40113
I20260809 19:24:45.668973 7899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.67:40113 every 8 connection(s)
I20260809 19:24:45.669263 7702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/data/info.pb
I20260809 19:24:45.669440 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 7702
I20260809 19:24:45.669567 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal/instance
I20260809 19:24:45.669793 9581 external_mini_cluster.cc:1428] Setting key 02f66e143d2f3d96068130ce32d803ec
I20260809 19:24:45.673970 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.669996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.67:51955 (local address 127.9.91.126:38815)
0809 19:24:45.670067 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.670070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.670364 (+ 294us) server_negotiation.cc:407] Connection header received
0809 19:24:45.670932 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.670933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.670946 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.670996 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:45.671587 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.672076 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.672938 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.673120 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.673161 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.673778 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:45.673779 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:45.673779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:45.673783 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:45.673797 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:45.673800 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:45.673839 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.673887 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.673912 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 19:24:45.674134 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.669953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:38815 (local address 127.9.91.67:51955)
0809 19:24:45.670227 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.670270 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.670833 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.671055 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.671058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.671066 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:24:45.671518 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.671522 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.672135 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.672136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.672881 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.672883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.673172 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.673172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.673268 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.673669 (+ 401us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 19:24:45.673669 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:24:45.673694 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 19:24:45.673713 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:24:45.673838 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:24:45.673840 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:24:45.673842 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.673883 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.673949 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":91,"threads_started":1}
I20260809 19:24:45.674595 7900 heartbeater.cc:347] Connected to a master server at 127.9.91.126:38815
I20260809 19:24:45.674670 7900 heartbeater.cc:464] Registering TS with master...
I20260809 19:24:45.674835 7900 heartbeater.cc:511] Master 127.9.91.126:38815 requested a full tablet report, sending...
I20260809 19:24:45.675185 7190 ts_manager.cc:194] Registered new tserver with Master: 1adbaac82137497da7d374229f08d75c (127.9.91.67:40113)
I20260809 19:24:45.675504 7190 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.91.67:51955
I20260809 19:24:45.681279 9581 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 19:24:45.685045 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.682533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44002 (local address 127.9.91.126:38815)
0809 19:24:45.682583 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.682585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.682658 (+ 73us) server_negotiation.cc:407] Connection header received
0809 19:24:45.682719 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.682720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.682731 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.682789 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:45.682938 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.683387 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.684113 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.684302 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.684351 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.684834 (+ 483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:45.684835 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:45.684835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:45.684839 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:45.684850 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:45.684853 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:45.684897 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.684950 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.684972 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:24:45.686233 7190 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 19:24:45.687713 7190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 19:24:45.693360 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.689709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.9.91.66:33507)
0809 19:24:45.689865 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.689869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.689880 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:24:45.689920 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.689922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.689937 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.690017 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:45.690166 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.691087 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.692227 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.692994 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.693061 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.693148 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.693249 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.693285 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260809 19:24:45.693357 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.689522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:33507 (local address 127.0.0.1:36354)
0809 19:24:45.689668 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.689705 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.689762 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.690041 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.690043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.690045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:45.690100 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.690102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.691165 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.691166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.692137 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.692139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.693073 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.693074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.693151 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.693164 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.693232 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.693280 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260809 19:24:45.693939 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.689522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:40741 (local address 127.0.0.1:41896)
0809 19:24:45.689999 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.690008 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.690079 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.690254 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.690255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.690257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:45.690299 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.690300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.691669 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.691670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.692676 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.692678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.693730 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.693731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.693792 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.693796 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.693839 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.693874 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":36,"thread_start_us":49,"threads_started":1}
I20260809 19:24:45.693969 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.689618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.9.91.65:40741)
0809 19:24:45.689914 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.689918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.690083 (+ 165us) server_negotiation.cc:407] Connection header received
0809 19:24:45.690140 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.690142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.690158 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.690374 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:45.690443 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.691580 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.692754 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.693655 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.693729 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.693816 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.693849 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.693889 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":89,"threads_started":1}
I20260809 19:24:45.694056 7583 tablet_service.cc:1467] Processing CreateTablet for tablet ba8b14116bb64ba1bb3979bddde14a83 (DEFAULT_TABLE table=test-table [id=87e5a183abfa44ad8b2b3a781142c66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:45.694231 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.689855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42010 (local address 127.9.91.67:40113)
0809 19:24:45.690002 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.690006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.690208 (+ 202us) server_negotiation.cc:407] Connection header received
0809 19:24:45.690251 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.690253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.690266 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.690337 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:45.690536 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.691603 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.692791 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.693720 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.693775 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.693840 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.694124 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.694162 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260809 19:24:45.694408 7583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba8b14116bb64ba1bb3979bddde14a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:45.694413 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.689699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40113 (local address 127.0.0.1:42010)
0809 19:24:45.690151 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.690158 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.690202 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.690426 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.690428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.690429 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:45.690462 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.690463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.691684 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.691685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.692713 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.692716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.693874 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.693875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.693945 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.693949 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.693996 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.694348 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":29,"thread_start_us":61,"threads_started":1}
I20260809 19:24:45.694770 7379 tablet_service.cc:1467] Processing CreateTablet for tablet ba8b14116bb64ba1bb3979bddde14a83 (DEFAULT_TABLE table=test-table [id=87e5a183abfa44ad8b2b3a781142c66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:45.695140 7379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba8b14116bb64ba1bb3979bddde14a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:45.695036 7797 tablet_service.cc:1467] Processing CreateTablet for tablet ba8b14116bb64ba1bb3979bddde14a83 (DEFAULT_TABLE table=test-table [id=87e5a183abfa44ad8b2b3a781142c66c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 19:24:45.695339 7797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba8b14116bb64ba1bb3979bddde14a83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:45.697162 7919 tablet_bootstrap.cc:492] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Bootstrap starting.
I20260809 19:24:45.697875 7919 tablet_bootstrap.cc:654] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:45.698172 7919 log.cc:824] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:45.698374 7923 tablet_bootstrap.cc:492] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9: Bootstrap starting.
I20260809 19:24:45.698841 7919 tablet_bootstrap.cc:492] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: No bootstrap required, opened a new log
I20260809 19:24:45.698902 7919 ts_tablet_manager.cc:1397] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:45.698966 7923 tablet_bootstrap.cc:654] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:45.699239 7923 log.cc:824] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:45.699537 7924 tablet_bootstrap.cc:492] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Bootstrap starting.
I20260809 19:24:45.699962 7923 tablet_bootstrap.cc:492] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9: No bootstrap required, opened a new log
I20260809 19:24:45.700033 7923 ts_tablet_manager.cc:1397] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 19:24:45.700313 7924 tablet_bootstrap.cc:654] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:45.700623 7924 log.cc:824] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:45.700976 7919 raft_consensus.cc:348] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.701220 7919 raft_consensus.cc:374] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:45.701278 7919 raft_consensus.cc:729] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbc6860699d441beaa573f32d2a7d697, State: Initialized, Role: FOLLOWER
I20260809 19:24:45.701395 7924 tablet_bootstrap.cc:492] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: No bootstrap required, opened a new log
I20260809 19:24:45.701393 7919 consensus_queue.cc:260] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.701469 7924 ts_tablet_manager.cc:1397] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 19:24:45.701632 7919 ts_tablet_manager.cc:1428] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 19:24:45.701756 7695 heartbeater.cc:503] Master 127.9.91.126:38815 was elected leader, sending a full tablet report...
I20260809 19:24:45.702682 7923 raft_consensus.cc:348] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.702839 7923 raft_consensus.cc:374] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:45.702873 7923 raft_consensus.cc:729] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597a3232f5a54e76bb938321339c32a9, State: Initialized, Role: FOLLOWER
I20260809 19:24:45.703020 7923 consensus_queue.cc:260] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.703276 7923 ts_tablet_manager.cc:1428] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 19:24:45.703316 7486 heartbeater.cc:503] Master 127.9.91.126:38815 was elected leader, sending a full tablet report...
I20260809 19:24:45.703891 7924 raft_consensus.cc:348] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.704046 7924 raft_consensus.cc:374] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:45.704092 7924 raft_consensus.cc:729] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1adbaac82137497da7d374229f08d75c, State: Initialized, Role: FOLLOWER
I20260809 19:24:45.704192 7924 consensus_queue.cc:260] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.704406 7924 ts_tablet_manager.cc:1428] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 19:24:45.704474 7900 heartbeater.cc:503] Master 127.9.91.126:38815 was elected leader, sending a full tablet report...
I20260809 19:24:45.726418 7934 raft_consensus.cc:482] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 19:24:45.726496 7934 raft_consensus.cc:504] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.726754 7934 leader_election.cc:288] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbc6860699d441beaa573f32d2a7d697 (127.9.91.66:33507), 1adbaac82137497da7d374229f08d75c (127.9.91.67:40113)
I20260809 19:24:45.730011 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.726954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:35165 (local address 127.9.91.66:33507)
0809 19:24:45.726989 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.726991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.727000 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:24:45.727040 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.727041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.727049 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.727088 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:45.727244 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.727919 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.729217 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.729727 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.729796 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.729804 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.729922 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.729955 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:24:45.730010 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.726869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.66:33507 (local address 127.9.91.65:35165)
0809 19:24:45.726937 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.726948 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.726988 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.727120 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.727120 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.727122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:45.727178 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.727179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.727984 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.727984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.729142 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.729144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.729801 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.729801 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.729853 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.729857 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.729908 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.729946 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 19:24:45.730115 7936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.726893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.67:40113 (local address 127.9.91.65:34981)
0809 19:24:45.727057 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.727065 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:24:45.727116 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:24:45.727302 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:24:45.727303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:24:45.727304 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 19:24:45.727365 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.727367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.728150 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.728151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.729234 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:24:45.729236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.729880 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.729881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:24:45.729943 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.729946 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 19:24:45.729993 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 19:24:45.730019 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":6,"thread_start_us":99,"threads_started":1}
I20260809 19:24:45.730186 7918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.726978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.91.65:34981 (local address 127.9.91.67:40113)
0809 19:24:45.727011 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.727012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.727120 (+ 108us) server_negotiation.cc:407] Connection header received
0809 19:24:45.727188 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.727189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.727196 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.727236 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 19:24:45.727430 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.728075 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.729302 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.729816 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.729869 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.729875 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.730006 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.730035 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:24:45.730441 7823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba8b14116bb64ba1bb3979bddde14a83" candidate_uuid: "597a3232f5a54e76bb938321339c32a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1adbaac82137497da7d374229f08d75c" is_pre_election: true
I20260809 19:24:45.730446 7616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba8b14116bb64ba1bb3979bddde14a83" candidate_uuid: "597a3232f5a54e76bb938321339c32a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc6860699d441beaa573f32d2a7d697" is_pre_election: true
I20260809 19:24:45.730542 7616 raft_consensus.cc:2454] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597a3232f5a54e76bb938321339c32a9 in term 0.
I20260809 19:24:45.730548 7823 raft_consensus.cc:2454] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 597a3232f5a54e76bb938321339c32a9 in term 0.
I20260809 19:24:45.730746 7311 leader_election.cc:302] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1adbaac82137497da7d374229f08d75c, 597a3232f5a54e76bb938321339c32a9; no voters:
I20260809 19:24:45.730873 7934 raft_consensus.cc:2781] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 19:24:45.730931 7934 raft_consensus.cc:482] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 19:24:45.730955 7934 raft_consensus.cc:3037] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:45.731513 7934 raft_consensus.cc:504] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.731630 7934 leader_election.cc:288] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [CANDIDATE]: Term 1 election: Requested vote from peers fbc6860699d441beaa573f32d2a7d697 (127.9.91.66:33507), 1adbaac82137497da7d374229f08d75c (127.9.91.67:40113)
I20260809 19:24:45.731767 7616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba8b14116bb64ba1bb3979bddde14a83" candidate_uuid: "597a3232f5a54e76bb938321339c32a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc6860699d441beaa573f32d2a7d697"
I20260809 19:24:45.731812 7823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba8b14116bb64ba1bb3979bddde14a83" candidate_uuid: "597a3232f5a54e76bb938321339c32a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1adbaac82137497da7d374229f08d75c"
I20260809 19:24:45.731835 7616 raft_consensus.cc:3037] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:45.731882 7823 raft_consensus.cc:3037] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:45.732470 7823 raft_consensus.cc:2454] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597a3232f5a54e76bb938321339c32a9 in term 1.
I20260809 19:24:45.732481 7616 raft_consensus.cc:2454] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 597a3232f5a54e76bb938321339c32a9 in term 1.
I20260809 19:24:45.732635 7309 leader_election.cc:302] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 597a3232f5a54e76bb938321339c32a9, fbc6860699d441beaa573f32d2a7d697; no voters:
I20260809 19:24:45.732725 7934 raft_consensus.cc:2781] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:45.732862 7934 raft_consensus.cc:686] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [term 1 LEADER]: Becoming Leader. State: Replica: 597a3232f5a54e76bb938321339c32a9, State: Running, Role: LEADER
I20260809 19:24:45.732956 7934 consensus_queue.cc:237] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } }
I20260809 19:24:45.733629 7189 catalog_manager.cc:5495] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 597a3232f5a54e76bb938321339c32a9 (127.9.91.65). New cstate: current_term: 1 leader_uuid: "597a3232f5a54e76bb938321339c32a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "597a3232f5a54e76bb938321339c32a9" member_type: VOTER last_known_addr { host: "127.9.91.65" port: 40741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 } health_report { overall_health: UNKNOWN } } }
I20260809 19:24:45.740589 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.738488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.9.91.65:40741)
0809 19:24:45.738526 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:45.738528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:45.738536 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:45.738593 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:45.738593 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:45.738607 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:45.738653 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:24:45.738731 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.739207 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.739990 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:45.740180 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:45.740225 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:45.740327 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:24:45.740410 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:24:45.740477 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:45.740509 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 19:24:45.740524 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:24:45.742075 7616 raft_consensus.cc:1261] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 1 FOLLOWER]: Refusing update from remote peer 597a3232f5a54e76bb938321339c32a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:45.742075 7823 raft_consensus.cc:1261] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 1 FOLLOWER]: Refusing update from remote peer 597a3232f5a54e76bb938321339c32a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:24:45.742419 7937 consensus_queue.cc:1035] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc6860699d441beaa573f32d2a7d697" member_type: VOTER last_known_addr { host: "127.9.91.66" port: 33507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:45.742550 7934 consensus_queue.cc:1035] T ba8b14116bb64ba1bb3979bddde14a83 P 597a3232f5a54e76bb938321339c32a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1adbaac82137497da7d374229f08d75c" member_type: VOTER last_known_addr { host: "127.9.91.67" port: 40113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:24:45.744948 7946 mvcc.cc:205] Tried to move back new op lower bound from 7316699077597429760 to 7316699077562736640. Current Snapshot: MvccSnapshot[applied={T|T < 7316699077597429760}]
I20260809 19:24:45.745748 7945 mvcc.cc:205] Tried to move back new op lower bound from 7316699077597429760 to 7316699077562736640. Current Snapshot: MvccSnapshot[applied={T|T < 7316699077597429760}]
I20260809 19:24:45.746949 7190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:45.747033 7190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 19:24:45.747640 7190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e488989f07f944f4866f8758570c6758: Sending DeleteTablet for 3 replicas of tablet ba8b14116bb64ba1bb3979bddde14a83
I20260809 19:24:45.747939 7379 tablet_service.cc:1514] Processing DeleteTablet for tablet ba8b14116bb64ba1bb3979bddde14a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:45 UTC) from {username='slave'} at 127.0.0.1:41896
I20260809 19:24:45.747951 7583 tablet_service.cc:1514] Processing DeleteTablet for tablet ba8b14116bb64ba1bb3979bddde14a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:45 UTC) from {username='slave'} at 127.0.0.1:36354
I20260809 19:24:45.748176 7797 tablet_service.cc:1514] Processing DeleteTablet for tablet ba8b14116bb64ba1bb3979bddde14a83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 19:24:45 UTC) from {username='slave'} at 127.0.0.1:42010
I20260809 19:24:45.748323 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 7285
I20260809 19:24:45.748438 7958 tablet_replica.cc:330] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: stopping tablet replica
I20260809 19:24:45.748512 7958 raft_consensus.cc:2229] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:45.748580 7958 raft_consensus.cc:2258] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:45.750593 7955 tablet_replica.cc:330] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: stopping tablet replica
I20260809 19:24:45.750669 7955 raft_consensus.cc:2229] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 19:24:45.750736 7955 raft_consensus.cc:2258] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 19:24:45.768803 7958 ts_tablet_manager.cc:1905] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:45.769945 7955 ts_tablet_manager.cc:1905] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 19:24:45.770213 7958 ts_tablet_manager.cc:1918] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:45.770278 7958 log.cc:1196] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-2/wal/wals/ba8b14116bb64ba1bb3979bddde14a83
I20260809 19:24:45.770500 7958 ts_tablet_manager.cc:1939] T ba8b14116bb64ba1bb3979bddde14a83 P 1adbaac82137497da7d374229f08d75c: Deleting consensus metadata
I20260809 19:24:45.770869 7163 catalog_manager.cc:4841] TS 1adbaac82137497da7d374229f08d75c (127.9.91.67:40113): tablet ba8b14116bb64ba1bb3979bddde14a83 (table test-table [id=87e5a183abfa44ad8b2b3a781142c66c]) successfully deleted
I20260809 19:24:45.771358 7955 ts_tablet_manager.cc:1918] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 19:24:45.771427 7955 log.cc:1196] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Deleting WAL directory at /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786303248613589-9581-0/minicluster-data/ts-1/wal/wals/ba8b14116bb64ba1bb3979bddde14a83
I20260809 19:24:45.771663 7955 ts_tablet_manager.cc:1939] T ba8b14116bb64ba1bb3979bddde14a83 P fbc6860699d441beaa573f32d2a7d697: Deleting consensus metadata
I20260809 19:24:45.772046 7161 catalog_manager.cc:4841] TS fbc6860699d441beaa573f32d2a7d697 (127.9.91.66:33507): tablet ba8b14116bb64ba1bb3979bddde14a83 (table test-table [id=87e5a183abfa44ad8b2b3a781142c66c]) successfully deleted
W20260809 19:24:45.774142 7163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.91.65:40741 (error 108)
I20260809 19:24:45.774380 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 7493
I20260809 19:24:45.774533 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:45.774427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.65:40741 (local address 127.0.0.1:41914)
0809 19:24:45.774472 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:24:45.774506 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.65:40741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 19:24:45.775688 7163 catalog_manager.cc:4584] TS 597a3232f5a54e76bb938321339c32a9 (127.9.91.65:40741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba8b14116bb64ba1bb3979bddde14a83: Network error: Client connection negotiation failed: client connection to 127.9.91.65:40741: connect: Connection refused (error 111)
I20260809 19:24:45.788516 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 7702
I20260809 19:24:45.799045 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 7134
2026-08-09T19:24:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (724 ms)
[----------] 6 tests from EncryptionPolicyTest (8534 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:45 dist-test-slave-1vmg krb5kdc[7965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:45 dist-test-slave-1vmg krb5kdc[7965](info): set up 2 sockets
Aug 09 19:24:45 dist-test-slave-1vmg krb5kdc[7965](info): commencing operation
krb5kdc: starting...
W20260809 19:24:46.916843 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:46 dist-test-slave-1vmg krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:47.025202 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:47.103443 7982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:47.103616 7982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:47.103638 7982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:47.104676 7982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:47.104708 7982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:47.104720 7982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:47.104740 7982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:47.104751 7982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:47.106139 7982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45967
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:47.106329 7982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:47.106498 7982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:47.110841 7990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:47.110814 7987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:47.111013 7982 server_base.cc:1034] running on GCE node
W20260809 19:24:47.110828 7992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:47.111279 7982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:47.111545 7982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:47.112684 7982 hybrid_clock.cc:648] HybridClock initialized: now 1786303487112667 us; error 29 us; skew 500 ppm
Aug 09 19:24:47 dist-test-slave-1vmg krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303487, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:47.115128 7982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:47.115490 7982 webserver.cc:460] Webserver started at http://127.0.0.1:42645/ using document root <none> and password file <none>
I20260809 19:24:47.115676 7982 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:47.115715 7982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:47.115800 7982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:47.116706 7982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "5f85565ab8ff4cdabef51de9877236f7"
format_stamp: "Formatted at 2026-08-09 19:24:47 on dist-test-slave-1vmg"
server_key: "c463cab856f98778e3e34a9d39b5c654"
server_key_iv: "f14da064be9ae409834ce99a1c8fedc3"
server_key_version: "encryptionkey@0"
I20260809 19:24:47.116976 7982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "5f85565ab8ff4cdabef51de9877236f7"
format_stamp: "Formatted at 2026-08-09 19:24:47 on dist-test-slave-1vmg"
server_key: "c463cab856f98778e3e34a9d39b5c654"
server_key_iv: "f14da064be9ae409834ce99a1c8fedc3"
server_key_version: "encryptionkey@0"
I20260809 19:24:47.118115 7982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:24:47.118727 8001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:47.118891 7982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:47.118994 7982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "5f85565ab8ff4cdabef51de9877236f7"
format_stamp: "Formatted at 2026-08-09 19:24:47 on dist-test-slave-1vmg"
server_key: "c463cab856f98778e3e34a9d39b5c654"
server_key_iv: "f14da064be9ae409834ce99a1c8fedc3"
server_key_version: "encryptionkey@0"
I20260809 19:24:47.119050 7982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:47.134618 7982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:47.136538 7982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:47.136646 7982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:47.140267 7982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45967
I20260809 19:24:47.140312 8087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45967 every 8 connection(s)
I20260809 19:24:47.140587 7982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:47.141467 8089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:47.143535 8089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7: Bootstrap starting.
I20260809 19:24:47.144074 8089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:47.144331 8089 log.cc:824] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:47.144948 8089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7: No bootstrap required, opened a new log
I20260809 19:24:47.146457 8089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f85565ab8ff4cdabef51de9877236f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45967 } }
I20260809 19:24:47.146593 8089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:47.146627 8089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f85565ab8ff4cdabef51de9877236f7, State: Initialized, Role: FOLLOWER
I20260809 19:24:47.146763 8089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f85565ab8ff4cdabef51de9877236f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45967 } }
I20260809 19:24:47.146842 8089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:47.146878 8089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:47.146943 8089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:47.147418 8089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f85565ab8ff4cdabef51de9877236f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45967 } }
I20260809 19:24:47.147509 8089 leader_election.cc:302] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f85565ab8ff4cdabef51de9877236f7; no voters:
I20260809 19:24:47.147668 8089 leader_election.cc:288] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:47.147699 8093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:47.147823 8093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [term 1 LEADER]: Becoming Leader. State: Replica: 5f85565ab8ff4cdabef51de9877236f7, State: Running, Role: LEADER
I20260809 19:24:47.147902 8089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:47.147915 8093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f85565ab8ff4cdabef51de9877236f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45967 } }
I20260809 19:24:47.148360 8095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f85565ab8ff4cdabef51de9877236f7. Latest consensus state: current_term: 1 leader_uuid: "5f85565ab8ff4cdabef51de9877236f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f85565ab8ff4cdabef51de9877236f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45967 } } }
I20260809 19:24:47.148442 8095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:47.148576 8094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f85565ab8ff4cdabef51de9877236f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f85565ab8ff4cdabef51de9877236f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45967 } } }
I20260809 19:24:47.148700 8094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:47.148883 8111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:47.149132 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 7982
I20260809 19:24:47.149262 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:47.149430 8111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:47.149466 9581 external_mini_cluster.cc:1428] Setting key ee49e0927cd3ad52c9c960b7139fec7e
I20260809 19:24:47.151511 8111 catalog_manager.cc:1347] Generated new cluster ID: adf3a843ff4f400e9a125908697b7ee0
I20260809 19:24:47.151564 8111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:24:47 dist-test-slave-1vmg krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:24:47.159309 8111 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:47.159770 8111 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:47.172199 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:47.149845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34059 (local address 127.0.0.1:45967)
0809 19:24:47.150009 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:47.150019 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:47.150186 (+ 167us) server_negotiation.cc:407] Connection header received
0809 19:24:47.150312 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:47.150319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:47.150373 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:47.150535 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:47.151208 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:47.152561 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:47.153648 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:47.154228 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:47.154323 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:47.157023 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:47.157028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:47.157028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:47.157040 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:47.158875 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:47.170966 (+ 12091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:47.170968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:47.170969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:47.171049 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:47.171305 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:47.171306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:47.171306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:47.171600 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:47.171740 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:47.171866 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 19:24:47.171966 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 19:24:47.173137 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:47.173465 8111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f85565ab8ff4cdabef51de9877236f7: Generated new TSK 0
I20260809 19:24:47.173615 8111 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:47.180469 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:47.174573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36063 (local address 127.0.0.1:45967)
0809 19:24:47.174610 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:47.174612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:47.174664 (+ 52us) server_negotiation.cc:407] Connection header received
0809 19:24:47.174730 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:47.174731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:47.174737 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:47.174779 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:47.175422 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:47.175887 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:47.176874 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:47.177063 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:47.177104 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:47.178296 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:47.178297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:47.178297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:47.178303 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:47.179471 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:47.179861 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:47.179862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:47.179862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:47.179905 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:47.180096 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:47.180096 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:47.180096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:47.180175 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:47.180235 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:47.180375 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 19:24:47.180418 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:24:47.182238 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 7982
W20260809 19:24:47.191035 7962 connection.cc:535] client connection to 127.0.0.1:45967 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45967): Connection reset by peer (error 104)
2026-08-09T19:24:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1383 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:47 dist-test-slave-1vmg krb5kdc[8139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:47 dist-test-slave-1vmg krb5kdc[8139](info): set up 2 sockets
Aug 09 19:24:47 dist-test-slave-1vmg krb5kdc[8139](info): commencing operation
krb5kdc: starting...
W20260809 19:24:48.323391 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:48 dist-test-slave-1vmg krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:48.429142 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:48.507529 8157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:48.507718 8157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:48.507740 8157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:48.508858 8157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:48.508898 8157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:48.508910 8157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:48.508924 8157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:48.508946 8157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:48.510337 8157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46069
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:48.510553 8157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:48.510742 8157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:48.515129 8164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:48.515134 8166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:48.515216 8162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:48.515225 8157 server_base.cc:1034] running on GCE node
I20260809 19:24:48.515576 8157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:48.515849 8157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:48.516990 8157 hybrid_clock.cc:648] HybridClock initialized: now 1786303488516975 us; error 29 us; skew 500 ppm
Aug 09 19:24:48 dist-test-slave-1vmg krb5kdc[8139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303488, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:48.519259 8157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:48.519615 8157 webserver.cc:460] Webserver started at http://127.0.0.1:35717/ using document root <none> and password file <none>
I20260809 19:24:48.519802 8157 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:48.519843 8157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:48.519930 8157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:48.520833 8157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "ef6dc0816c974293a78d7ab0c16e9a83"
format_stamp: "Formatted at 2026-08-09 19:24:48 on dist-test-slave-1vmg"
server_key: "80924b0cb0f595078b083a91ca28dbe2"
server_key_iv: "2d2480318dc55832d9aa19ea614dd229"
server_key_version: "encryptionkey@0"
I20260809 19:24:48.521112 8157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "ef6dc0816c974293a78d7ab0c16e9a83"
format_stamp: "Formatted at 2026-08-09 19:24:48 on dist-test-slave-1vmg"
server_key: "80924b0cb0f595078b083a91ca28dbe2"
server_key_iv: "2d2480318dc55832d9aa19ea614dd229"
server_key_version: "encryptionkey@0"
I20260809 19:24:48.522230 8157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:24:48.522826 8174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:48.522989 8157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:48.523072 8157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "ef6dc0816c974293a78d7ab0c16e9a83"
format_stamp: "Formatted at 2026-08-09 19:24:48 on dist-test-slave-1vmg"
server_key: "80924b0cb0f595078b083a91ca28dbe2"
server_key_iv: "2d2480318dc55832d9aa19ea614dd229"
server_key_version: "encryptionkey@0"
I20260809 19:24:48.523132 8157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:48.535311 8157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:48.536830 8157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:48.536952 8157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:48.540694 8157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46069
I20260809 19:24:48.540723 8259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46069 every 8 connection(s)
I20260809 19:24:48.541011 8157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:48.541853 8260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:48.542871 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 8157
I20260809 19:24:48.543006 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:48.543179 9581 external_mini_cluster.cc:1428] Setting key aab861269adfbf2da12210bbe002f1c8
I20260809 19:24:48.544304 8260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83: Bootstrap starting.
I20260809 19:24:48.544948 8260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:48.545275 8260 log.cc:824] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:48.545974 8260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83: No bootstrap required, opened a new log
I20260809 19:24:48.547996 8260 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46069 } }
I20260809 19:24:48.548166 8260 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:48.548265 8260 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef6dc0816c974293a78d7ab0c16e9a83, State: Initialized, Role: FOLLOWER
I20260809 19:24:48.548419 8260 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46069 } }
I20260809 19:24:48.548513 8260 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:48.548552 8260 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:48.548563 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:48.543594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44697 (local address 127.0.0.1:46069)
0809 19:24:48.543766 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:48.543775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:48.543808 (+ 33us) server_negotiation.cc:407] Connection header received
0809 19:24:48.544271 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:48.544280 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:48.544345 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:48.544487 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:48.544638 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:48.545858 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:48.546623 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:48.547255 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:48.547376 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:48.547906 (+ 530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:48.547909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:48.547909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:48.548090 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:48.548125 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:48.548139 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:48.548227 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:48.548308 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 19:24:48.548336 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260809 19:24:48.548597 8260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:48.549167 8260 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46069 } }
I20260809 19:24:48.549273 8260 leader_election.cc:302] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef6dc0816c974293a78d7ab0c16e9a83; no voters:
I20260809 19:24:48.549431 8260 leader_election.cc:288] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:48.549455 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:48.549463 8273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:48.549633 8273 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [term 1 LEADER]: Becoming Leader. State: Replica: ef6dc0816c974293a78d7ab0c16e9a83, State: Running, Role: LEADER
I20260809 19:24:48.549662 8260 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:48.549760 8273 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46069 } }
I20260809 19:24:48.550156 8275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef6dc0816c974293a78d7ab0c16e9a83. Latest consensus state: current_term: 1 leader_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46069 } } }
I20260809 19:24:48.550236 8275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:48.550379 8274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6dc0816c974293a78d7ab0c16e9a83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46069 } } }
I20260809 19:24:48.550439 8274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83 [sys.catalog]: This master's current role is: LEADER
W20260809 19:24:48.551488 8301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:24:48.551548 8301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:24:48.551589 8281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:48.551755 8281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:48.553115 8281 catalog_manager.cc:1347] Generated new cluster ID: 2ad56cd99247452493194536a7773932
I20260809 19:24:48.553174 8281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:48.556290 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:48.551458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40635 (local address 127.0.0.1:46069)
0809 19:24:48.551488 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:48.551490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:48.551678 (+ 188us) server_negotiation.cc:407] Connection header received
0809 19:24:48.551759 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:48.551760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:48.551770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:48.551815 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:48.551975 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:48.552516 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:48.555273 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:48.555494 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:48.555554 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:48.556078 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:48.556079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:48.556079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:48.556083 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:48.556095 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:48.556099 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:48.556148 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:48.556217 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:24:48.556225 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 19:24:48.574743 8281 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:48.575438 8281 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:48.580856 8281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef6dc0816c974293a78d7ab0c16e9a83: Generated new TSK 0
I20260809 19:24:48.581009 8281 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:48.583985 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 8157
W20260809 19:24:48.593753 8137 connection.cc:535] client connection to 127.0.0.1:46069 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46069): Connection reset by peer (error 104)
2026-08-09T19:24:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:48 dist-test-slave-1vmg krb5kdc[8308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:48 dist-test-slave-1vmg krb5kdc[8308](info): set up 2 sockets
Aug 09 19:24:48 dist-test-slave-1vmg krb5kdc[8308](info): commencing operation
krb5kdc: starting...
W20260809 19:24:49.703249 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:49 dist-test-slave-1vmg krb5kdc[8308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:49.808851 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:49.885969 8329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:49.886149 8329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:49.886179 8329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:49.887295 8329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:49.887332 8329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:49.887351 8329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:49.887367 8329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:49.887383 8329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:49.888810 8329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:49.889034 8329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:49.889237 8329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:49.893567 8341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:49.893702 8336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:49.893612 8338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:49.893760 8329 server_base.cc:1034] running on GCE node
I20260809 19:24:49.894050 8329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:49.894320 8329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:49.895450 8329 hybrid_clock.cc:648] HybridClock initialized: now 1786303489895438 us; error 30 us; skew 500 ppm
Aug 09 19:24:49 dist-test-slave-1vmg krb5kdc[8308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303489, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:49.901376 8329 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:49.901813 8329 webserver.cc:460] Webserver started at http://127.0.0.1:40793/ using document root <none> and password file <none>
I20260809 19:24:49.902036 8329 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:49.902084 8329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:49.902222 8329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:49.903303 8329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "559e5c50dfe44c9db02d08693259964e"
format_stamp: "Formatted at 2026-08-09 19:24:49 on dist-test-slave-1vmg"
server_key: "a6610561c49cc85ce55bf404a3c4561d"
server_key_iv: "c75ca102ae116ff281d1bc5796fdda66"
server_key_version: "encryptionkey@0"
I20260809 19:24:49.903620 8329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "559e5c50dfe44c9db02d08693259964e"
format_stamp: "Formatted at 2026-08-09 19:24:49 on dist-test-slave-1vmg"
server_key: "a6610561c49cc85ce55bf404a3c4561d"
server_key_iv: "c75ca102ae116ff281d1bc5796fdda66"
server_key_version: "encryptionkey@0"
I20260809 19:24:49.904820 8329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:24:49.905476 8353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:49.905627 8329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:49.905709 8329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "559e5c50dfe44c9db02d08693259964e"
format_stamp: "Formatted at 2026-08-09 19:24:49 on dist-test-slave-1vmg"
server_key: "a6610561c49cc85ce55bf404a3c4561d"
server_key_iv: "c75ca102ae116ff281d1bc5796fdda66"
server_key_version: "encryptionkey@0"
I20260809 19:24:49.905774 8329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:49.908885 8329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:49.910542 8329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:49.910668 8329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:49.914384 8329 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37477
I20260809 19:24:49.914428 8440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37477 every 8 connection(s)
I20260809 19:24:49.914700 8329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:49.915632 8441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:49.917785 8441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e: Bootstrap starting.
I20260809 19:24:49.918340 8441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:49.918632 8441 log.cc:824] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:49.919279 8441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e: No bootstrap required, opened a new log
I20260809 19:24:49.920871 8441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559e5c50dfe44c9db02d08693259964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260809 19:24:49.921017 8441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:49.921060 8441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e5c50dfe44c9db02d08693259964e, State: Initialized, Role: FOLLOWER
I20260809 19:24:49.921202 8441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559e5c50dfe44c9db02d08693259964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260809 19:24:49.921284 8441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:49.921325 8441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:49.921375 8441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:49.921904 8441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559e5c50dfe44c9db02d08693259964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260809 19:24:49.922010 8441 leader_election.cc:302] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 559e5c50dfe44c9db02d08693259964e; no voters:
I20260809 19:24:49.922175 8441 leader_election.cc:288] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:49.922204 8446 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:49.922425 8441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:49.922484 8446 raft_consensus.cc:686] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [term 1 LEADER]: Becoming Leader. State: Replica: 559e5c50dfe44c9db02d08693259964e, State: Running, Role: LEADER
I20260809 19:24:49.922616 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 8329
I20260809 19:24:49.922631 8446 consensus_queue.cc:237] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559e5c50dfe44c9db02d08693259964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } }
I20260809 19:24:49.922751 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:49.922991 9581 external_mini_cluster.cc:1428] Setting key 8c4b2f4beeb6e276cf71de2e89ee7c37
I20260809 19:24:49.922969 8447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "559e5c50dfe44c9db02d08693259964e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559e5c50dfe44c9db02d08693259964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } } }
I20260809 19:24:49.923038 8447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:49.923199 8448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 559e5c50dfe44c9db02d08693259964e. Latest consensus state: current_term: 1 leader_uuid: "559e5c50dfe44c9db02d08693259964e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "559e5c50dfe44c9db02d08693259964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37477 } } }
I20260809 19:24:49.923257 8448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:49.923616 8456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:49.924135 8456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:49.925858 8456 catalog_manager.cc:1347] Generated new cluster ID: f6b11a8d2f574551807730c8d1604aa0
I20260809 19:24:49.925897 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:49.923713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44653 (local address 127.0.0.1:37477)
0809 19:24:49.923850 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:49.923859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:49.923999 (+ 140us) server_negotiation.cc:407] Connection header received
0809 19:24:49.924155 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:49.924162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:49.924193 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:49.924335 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:49.925416 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:49.925419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:49.925420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:49.925426 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:49.925472 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:49.925488 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:49.925532 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:49.925655 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 19:24:49.925697 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":64,"threads_started":1}
I20260809 19:24:49.925915 8456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:49.926815 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:49.929309 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:49.928281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55985 (local address 127.0.0.1:37477)
0809 19:24:49.928313 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:49.928314 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:49.928431 (+ 117us) server_negotiation.cc:407] Connection header received
0809 19:24:49.928490 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:49.928497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:49.928502 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:49.928570 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:49.929091 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:49.929092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:49.929092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:49.929096 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:49.929109 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:49.929113 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:49.929170 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:49.929225 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 19:24:49.929237 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:24:49.930832 8456 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:49.930994 8456 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:49.934685 8456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 559e5c50dfe44c9db02d08693259964e: Generated new TSK 0
I20260809 19:24:49.934830 8456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:49.940634 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 8329
2026-08-09T19:24:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:49 dist-test-slave-1vmg krb5kdc[8489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:49 dist-test-slave-1vmg krb5kdc[8489](info): set up 2 sockets
Aug 09 19:24:49 dist-test-slave-1vmg krb5kdc[8489](info): commencing operation
krb5kdc: starting...
W20260809 19:24:51.063827 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:51 dist-test-slave-1vmg krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:51.171348 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:51.248939 8507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:51.249116 8507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:51.249137 8507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:51.250228 8507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:51.250268 8507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:51.250281 8507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:51.250305 8507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:51.250327 8507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:51.251775 8507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:51.251984 8507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:51.252177 8507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:51.256552 8516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:51.256611 8518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:51.256549 8515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:51.256717 8507 server_base.cc:1034] running on GCE node
I20260809 19:24:51.256976 8507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:51.257247 8507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:51.258383 8507 hybrid_clock.cc:648] HybridClock initialized: now 1786303491258366 us; error 29 us; skew 500 ppm
Aug 09 19:24:51 dist-test-slave-1vmg krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303491, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:51.260673 8507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:51.261112 8507 webserver.cc:460] Webserver started at http://127.0.0.1:40907/ using document root <none> and password file <none>
I20260809 19:24:51.261317 8507 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:51.261363 8507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:51.261485 8507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:51.262418 8507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "ff542687d7ee427a8638055d6e8ed77a"
format_stamp: "Formatted at 2026-08-09 19:24:51 on dist-test-slave-1vmg"
server_key: "ec235c743e32a68caa4211e39e72f7cc"
server_key_iv: "6cbd97677f9ad0fc92a054a96be5e1e9"
server_key_version: "encryptionkey@0"
I20260809 19:24:51.262734 8507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "ff542687d7ee427a8638055d6e8ed77a"
format_stamp: "Formatted at 2026-08-09 19:24:51 on dist-test-slave-1vmg"
server_key: "ec235c743e32a68caa4211e39e72f7cc"
server_key_iv: "6cbd97677f9ad0fc92a054a96be5e1e9"
server_key_version: "encryptionkey@0"
I20260809 19:24:51.263926 8507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:24:51.264570 8529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:51.264725 8507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:51.264801 8507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "ff542687d7ee427a8638055d6e8ed77a"
format_stamp: "Formatted at 2026-08-09 19:24:51 on dist-test-slave-1vmg"
server_key: "ec235c743e32a68caa4211e39e72f7cc"
server_key_iv: "6cbd97677f9ad0fc92a054a96be5e1e9"
server_key_version: "encryptionkey@0"
I20260809 19:24:51.264868 8507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:51.281550 8507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:51.283434 8507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:51.283550 8507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:51.287333 8507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45391
I20260809 19:24:51.287361 8615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45391 every 8 connection(s)
I20260809 19:24:51.287636 8507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:51.288486 8617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:51.290452 8617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a: Bootstrap starting.
I20260809 19:24:51.290983 8617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:51.291251 8617 log.cc:824] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:51.291837 8617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a: No bootstrap required, opened a new log
I20260809 19:24:51.293313 8617 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff542687d7ee427a8638055d6e8ed77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45391 } }
I20260809 19:24:51.293437 8617 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:51.293457 8617 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff542687d7ee427a8638055d6e8ed77a, State: Initialized, Role: FOLLOWER
I20260809 19:24:51.293553 8617 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff542687d7ee427a8638055d6e8ed77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45391 } }
I20260809 19:24:51.293599 8617 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:51.293624 8617 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:51.293653 8617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:51.294092 8617 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff542687d7ee427a8638055d6e8ed77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45391 } }
I20260809 19:24:51.294173 8617 leader_election.cc:302] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff542687d7ee427a8638055d6e8ed77a; no voters:
I20260809 19:24:51.294297 8617 leader_election.cc:288] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:51.294337 8622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:51.294500 8622 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [term 1 LEADER]: Becoming Leader. State: Replica: ff542687d7ee427a8638055d6e8ed77a, State: Running, Role: LEADER
I20260809 19:24:51.294503 8617 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:51.294662 8622 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff542687d7ee427a8638055d6e8ed77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45391 } }
I20260809 19:24:51.295076 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 8507
I20260809 19:24:51.295138 8623 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff542687d7ee427a8638055d6e8ed77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff542687d7ee427a8638055d6e8ed77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45391 } } }
I20260809 19:24:51.295199 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:51.295209 8623 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:51.295370 8624 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff542687d7ee427a8638055d6e8ed77a. Latest consensus state: current_term: 1 leader_uuid: "ff542687d7ee427a8638055d6e8ed77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff542687d7ee427a8638055d6e8ed77a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45391 } } }
I20260809 19:24:51.295394 9581 external_mini_cluster.cc:1428] Setting key c609765e14188ca680683bc9b458dde6
I20260809 19:24:51.295421 8624 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:51.295773 8635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:51.296303 8635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:51.298502 8635 catalog_manager.cc:1347] Generated new cluster ID: 57fc7155dd9d4691a0e5ff44d0ef421b
I20260809 19:24:51.298555 8635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:24:51 dist-test-slave-1vmg krb5kdc[8489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:24:51.306432 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:51.296093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49903 (local address 127.0.0.1:45391)
0809 19:24:51.296285 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:51.296295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:51.296326 (+ 31us) server_negotiation.cc:407] Connection header received
0809 19:24:51.296431 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:51.296437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:51.296489 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:51.296649 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:51.297365 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:51.298382 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:51.299429 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:51.300058 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:51.300144 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:51.302861 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:51.302865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:51.302865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:51.302877 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:51.304709 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:51.305108 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:51.305111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:51.305112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:51.305234 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:51.305459 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:51.305460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:51.305460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:51.305744 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:51.305899 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:51.305969 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 19:24:51.306204 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260809 19:24:51.306666 8635 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:51.307324 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:51.307364 8635 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:51.313730 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:51.308537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37267 (local address 127.0.0.1:45391)
0809 19:24:51.308570 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:51.308572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:51.308668 (+ 96us) server_negotiation.cc:407] Connection header received
0809 19:24:51.308721 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:51.308722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:51.308731 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:51.308774 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:51.309241 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:51.309715 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:51.310375 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:51.310597 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:51.310649 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:51.311649 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:51.311650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:51.311650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:51.311657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:51.312810 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:51.313094 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:51.313095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:51.313095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:51.313146 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:51.313303 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:51.313304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:51.313304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:51.313400 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:51.313476 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:51.313578 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 19:24:51.313653 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:24:51.316929 8635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff542687d7ee427a8638055d6e8ed77a: Generated new TSK 0
I20260809 19:24:51.317080 8635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:51.320698 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 8507
2026-08-09T19:24:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:51 dist-test-slave-1vmg krb5kdc[8664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:51 dist-test-slave-1vmg krb5kdc[8664](info): set up 2 sockets
Aug 09 19:24:51 dist-test-slave-1vmg krb5kdc[8664](info): commencing operation
krb5kdc: starting...
W20260809 19:24:52.452111 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:52 dist-test-slave-1vmg krb5kdc[8664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:52.576999 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:52.656430 8687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:52.656625 8687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:52.656659 8687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:52.657769 8687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:52.657816 8687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:52.657840 8687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:52.657859 8687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:52.657878 8687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:52.659370 8687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:52.659602 8687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:52.659816 8687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:52.664254 8698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:52.664414 8687 server_base.cc:1034] running on GCE node
W20260809 19:24:52.664259 8694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:52.664331 8695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:52.664674 8687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:52.664909 8687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:52.666031 8687 hybrid_clock.cc:648] HybridClock initialized: now 1786303492666023 us; error 26 us; skew 500 ppm
Aug 09 19:24:52 dist-test-slave-1vmg krb5kdc[8664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303492, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:52.668251 8687 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:52.668618 8687 webserver.cc:460] Webserver started at http://127.0.0.1:44487/ using document root <none> and password file <none>
I20260809 19:24:52.668797 8687 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:52.668838 8687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:52.668958 8687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:52.669883 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "99ec8304fa9f4310ad8105a1c1b34eee"
format_stamp: "Formatted at 2026-08-09 19:24:52 on dist-test-slave-1vmg"
server_key: "238d547ac70cc74bb002bd1e3d179a0a"
server_key_iv: "69a09141da1de76a416b2a7297cddc5d"
server_key_version: "encryptionkey@0"
I20260809 19:24:52.670195 8687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "99ec8304fa9f4310ad8105a1c1b34eee"
format_stamp: "Formatted at 2026-08-09 19:24:52 on dist-test-slave-1vmg"
server_key: "238d547ac70cc74bb002bd1e3d179a0a"
server_key_iv: "69a09141da1de76a416b2a7297cddc5d"
server_key_version: "encryptionkey@0"
I20260809 19:24:52.671473 8687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:24:52.672086 8711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:52.672219 8687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:52.672293 8687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "99ec8304fa9f4310ad8105a1c1b34eee"
format_stamp: "Formatted at 2026-08-09 19:24:52 on dist-test-slave-1vmg"
server_key: "238d547ac70cc74bb002bd1e3d179a0a"
server_key_iv: "69a09141da1de76a416b2a7297cddc5d"
server_key_version: "encryptionkey@0"
I20260809 19:24:52.672355 8687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:52.691946 8687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:52.693446 8687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:52.693562 8687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:52.697258 8687 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39111
I20260809 19:24:52.697292 8790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39111 every 8 connection(s)
I20260809 19:24:52.697568 8687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:52.698417 8791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:52.700520 8791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee: Bootstrap starting.
I20260809 19:24:52.700834 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 8687
I20260809 19:24:52.700940 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:52.701095 9581 external_mini_cluster.cc:1428] Setting key 09a77e50ed26ed619a289734173db020
I20260809 19:24:52.701105 8791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:52.701372 8791 log.cc:824] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:52.702108 8791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee: No bootstrap required, opened a new log
I20260809 19:24:52.703851 8791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39111 } }
I20260809 19:24:52.703974 8791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:52.704005 8791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ec8304fa9f4310ad8105a1c1b34eee, State: Initialized, Role: FOLLOWER
I20260809 19:24:52.704125 8791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39111 } }
I20260809 19:24:52.704187 8791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:52.704218 8791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:52.704264 8791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:52.704754 8791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39111 } }
I20260809 19:24:52.704854 8791 leader_election.cc:302] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99ec8304fa9f4310ad8105a1c1b34eee; no voters:
I20260809 19:24:52.705000 8791 leader_election.cc:288] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:52.705027 8796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:52.705147 8796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [term 1 LEADER]: Becoming Leader. State: Replica: 99ec8304fa9f4310ad8105a1c1b34eee, State: Running, Role: LEADER
I20260809 19:24:52.705248 8791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:52.705247 8796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39111 } }
I20260809 19:24:52.705799 8800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39111 } } }
I20260809 19:24:52.705899 8800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:52.706053 8801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99ec8304fa9f4310ad8105a1c1b34eee. Latest consensus state: current_term: 1 leader_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99ec8304fa9f4310ad8105a1c1b34eee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39111 } } }
I20260809 19:24:52.706104 8801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:52.706327 8811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:52.706493 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:52.701535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60617 (local address 127.0.0.1:39111)
0809 19:24:52.701700 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:52.701706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:52.701729 (+ 23us) server_negotiation.cc:407] Connection header received
0809 19:24:52.702141 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:52.702147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:52.702190 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:52.702326 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:52.702522 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:52.703409 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:52.704180 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:52.704626 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:52.704720 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:52.705391 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:52.705395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:52.705395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:52.705606 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:52.705642 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:52.705657 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:52.705724 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:52.706192 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 19:24:52.706258 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260809 19:24:52.706822 8811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:52.707372 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:52.708331 8811 catalog_manager.cc:1347] Generated new cluster ID: b11d9919ae9c423fa05f340ab767832f
I20260809 19:24:52.708379 8811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:52.712248 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:52.708798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43863 (local address 127.0.0.1:39111)
0809 19:24:52.708833 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:52.708834 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:52.708955 (+ 121us) server_negotiation.cc:407] Connection header received
0809 19:24:52.709047 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:52.709048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:52.709058 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:52.709086 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:52.709283 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:52.709986 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:52.711024 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:52.711326 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:52.711368 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:52.711993 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:52.711995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:52.711995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:52.711999 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:52.712011 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:52.712015 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:52.712042 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:52.712159 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 19:24:52.712180 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:24:52.732492 8811 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:52.733176 8811 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:52.736016 8811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99ec8304fa9f4310ad8105a1c1b34eee: Generated new TSK 0
I20260809 19:24:52.736171 8811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:52.739495 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 8687
2026-08-09T19:24:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:52 dist-test-slave-1vmg krb5kdc[8846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:52 dist-test-slave-1vmg krb5kdc[8846](info): set up 2 sockets
Aug 09 19:24:52 dist-test-slave-1vmg krb5kdc[8846](info): commencing operation
krb5kdc: starting...
W20260809 19:24:53.873232 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:53 dist-test-slave-1vmg krb5kdc[8846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:53.982417 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:54.060230 8869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:54.060413 8869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:54.060443 8869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:54.061513 8869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:54.061555 8869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:54.061575 8869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:54.061599 8869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:54.061621 8869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:54.063050 8869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:54.063272 8869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:54.063458 8869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:54.067911 8876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:54.067996 8869 server_base.cc:1034] running on GCE node
W20260809 19:24:54.067929 8879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:54.067931 8877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:54.068362 8869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:54.068614 8869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:54.069751 8869 hybrid_clock.cc:648] HybridClock initialized: now 1786303494069735 us; error 28 us; skew 500 ppm
Aug 09 19:24:54 dist-test-slave-1vmg krb5kdc[8846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303494, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:54.071995 8869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:54.072402 8869 webserver.cc:460] Webserver started at http://127.0.0.1:34411/ using document root <none> and password file <none>
I20260809 19:24:54.072588 8869 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:54.072634 8869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:54.072751 8869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:54.073661 8869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "2acd9a8f434747208e0bc4f65bf6a09e"
format_stamp: "Formatted at 2026-08-09 19:24:54 on dist-test-slave-1vmg"
server_key: "a7ad36c666878ac354a60efcab010601"
server_key_iv: "998c519f9c8cb759a6bd6ca1559eae7c"
server_key_version: "encryptionkey@0"
I20260809 19:24:54.073966 8869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "2acd9a8f434747208e0bc4f65bf6a09e"
format_stamp: "Formatted at 2026-08-09 19:24:54 on dist-test-slave-1vmg"
server_key: "a7ad36c666878ac354a60efcab010601"
server_key_iv: "998c519f9c8cb759a6bd6ca1559eae7c"
server_key_version: "encryptionkey@0"
I20260809 19:24:54.075165 8869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:24:54.075804 8889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:54.075949 8869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:54.076041 8869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "2acd9a8f434747208e0bc4f65bf6a09e"
format_stamp: "Formatted at 2026-08-09 19:24:54 on dist-test-slave-1vmg"
server_key: "a7ad36c666878ac354a60efcab010601"
server_key_iv: "998c519f9c8cb759a6bd6ca1559eae7c"
server_key_version: "encryptionkey@0"
I20260809 19:24:54.076093 8869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:54.079090 8869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:54.080598 8869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:54.080693 8869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:54.084337 8869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41069
I20260809 19:24:54.084368 8974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41069 every 8 connection(s)
I20260809 19:24:54.084622 8869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:54.085479 8975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:54.086030 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 8869
I20260809 19:24:54.086158 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:54.086329 9581 external_mini_cluster.cc:1428] Setting key 8d871cec4cada0e97e8c24d6812b2c2b
I20260809 19:24:54.088174 8975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e: Bootstrap starting.
I20260809 19:24:54.088405 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:54.086792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39053 (local address 127.0.0.1:41069)
0809 19:24:54.086963 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:54.086971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:54.086995 (+ 24us) server_negotiation.cc:407] Connection header received
0809 19:24:54.087360 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:54.087366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:54.087387 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:54.087488 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:54.088056 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:54.088059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:54.088059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:54.088062 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:54.088095 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:54.088105 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:54.088156 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:54.088234 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 19:24:54.088270 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260809 19:24:54.088860 8975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:54.089159 8975 log.cc:824] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:54.089279 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:54.089808 8975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e: No bootstrap required, opened a new log
I20260809 19:24:54.091451 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:54.090591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60209 (local address 127.0.0.1:41069)
0809 19:24:54.090632 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:54.090634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:54.090713 (+ 79us) server_negotiation.cc:407] Connection header received
0809 19:24:54.090761 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:54.090762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:54.090766 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:54.090809 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:54.091284 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:54.091285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:54.091285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:54.091289 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:54.091300 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:54.091304 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:54.091348 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:54.091387 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 19:24:54.091398 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 19:24:54.091806 8975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41069 } }
I20260809 19:24:54.091959 8975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:54.092002 8975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2acd9a8f434747208e0bc4f65bf6a09e, State: Initialized, Role: FOLLOWER
I20260809 19:24:54.092111 8975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41069 } }
I20260809 19:24:54.092191 8975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:54.092226 8975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:54.092270 8975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:54.092746 8975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41069 } }
I20260809 19:24:54.092832 8975 leader_election.cc:302] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2acd9a8f434747208e0bc4f65bf6a09e; no voters:
I20260809 19:24:54.092986 8975 leader_election.cc:288] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:54.093029 8986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:54.093170 8986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [term 1 LEADER]: Becoming Leader. State: Replica: 2acd9a8f434747208e0bc4f65bf6a09e, State: Running, Role: LEADER
I20260809 19:24:54.093210 8975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:54.093281 8986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41069 } }
I20260809 19:24:54.093595 8987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41069 } } }
I20260809 19:24:54.093705 8987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:54.093600 8988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2acd9a8f434747208e0bc4f65bf6a09e. Latest consensus state: current_term: 1 leader_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2acd9a8f434747208e0bc4f65bf6a09e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41069 } } }
I20260809 19:24:54.093801 8988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:54.094137 8997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:54.094671 8997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:54.096004 8997 catalog_manager.cc:1347] Generated new cluster ID: 7ba4b95d60bd4ca888d7d17f423e6df8
I20260809 19:24:54.096062 8997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:54.118135 8997 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:54.118290 8997 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:54.129696 8997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2acd9a8f434747208e0bc4f65bf6a09e: Generated new TSK 0
I20260809 19:24:54.129839 8997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:54.139819 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 8869
2026-08-09T19:24:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:54 dist-test-slave-1vmg krb5kdc[9013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:54 dist-test-slave-1vmg krb5kdc[9013](info): set up 2 sockets
Aug 09 19:24:54 dist-test-slave-1vmg krb5kdc[9013](info): commencing operation
krb5kdc: starting...
W20260809 19:24:55.270645 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:55 dist-test-slave-1vmg krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:55.378861 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42127
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:55.456290 9036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:55.456465 9036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:55.456496 9036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:55.457594 9036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:55.457634 9036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:55.457652 9036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:55.457667 9036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:55.457684 9036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:55.459124 9036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:55.459344 9036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:55.459539 9036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:55.463912 9047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:55.463905 9048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:55.463932 9051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:55.464293 9036 server_base.cc:1034] running on GCE node
I20260809 19:24:55.464473 9036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:55.464723 9036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:55.465844 9036 hybrid_clock.cc:648] HybridClock initialized: now 1786303495465836 us; error 27 us; skew 500 ppm
Aug 09 19:24:55 dist-test-slave-1vmg krb5kdc[9013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303495, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:55.470801 9036 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:55.471194 9036 webserver.cc:460] Webserver started at http://127.0.0.1:35785/ using document root <none> and password file <none>
I20260809 19:24:55.471379 9036 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:55.471416 9036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:55.471503 9036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:55.472397 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "e473682ab25f42d8b91571fdda6ec179"
format_stamp: "Formatted at 2026-08-09 19:24:55 on dist-test-slave-1vmg"
server_key: "24861082e13022814890864fc61fc980"
server_key_iv: "c3f68d574299282fe49e307ec9eed767"
server_key_version: "encryptionkey@0"
I20260809 19:24:55.472672 9036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "e473682ab25f42d8b91571fdda6ec179"
format_stamp: "Formatted at 2026-08-09 19:24:55 on dist-test-slave-1vmg"
server_key: "24861082e13022814890864fc61fc980"
server_key_iv: "c3f68d574299282fe49e307ec9eed767"
server_key_version: "encryptionkey@0"
I20260809 19:24:55.473786 9036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 19:24:55.474380 9061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:55.474519 9036 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:55.474601 9036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "e473682ab25f42d8b91571fdda6ec179"
format_stamp: "Formatted at 2026-08-09 19:24:55 on dist-test-slave-1vmg"
server_key: "24861082e13022814890864fc61fc980"
server_key_iv: "c3f68d574299282fe49e307ec9eed767"
server_key_version: "encryptionkey@0"
I20260809 19:24:55.474664 9036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:55.481609 9036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:55.483193 9036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:55.483312 9036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:55.486969 9036 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42127
I20260809 19:24:55.487004 9142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42127 every 8 connection(s)
I20260809 19:24:55.487293 9036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:55.488118 9143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:55.490048 9143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179: Bootstrap starting.
I20260809 19:24:55.490561 9143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:55.490818 9143 log.cc:824] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:55.491444 9143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179: No bootstrap required, opened a new log
I20260809 19:24:55.492475 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9036
I20260809 19:24:55.492604 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:55.492789 9581 external_mini_cluster.cc:1428] Setting key 0eac3aa8cb1a08ab62baac65ec35e3aa
I20260809 19:24:55.493203 9143 raft_consensus.cc:348] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e473682ab25f42d8b91571fdda6ec179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260809 19:24:55.493359 9143 raft_consensus.cc:374] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:55.493482 9143 raft_consensus.cc:729] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e473682ab25f42d8b91571fdda6ec179, State: Initialized, Role: FOLLOWER
I20260809 19:24:55.493629 9143 consensus_queue.cc:260] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e473682ab25f42d8b91571fdda6ec179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260809 19:24:55.493700 9143 raft_consensus.cc:388] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:55.493731 9143 raft_consensus.cc:482] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:55.493778 9143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:55.494563 9143 raft_consensus.cc:504] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e473682ab25f42d8b91571fdda6ec179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260809 19:24:55.494678 9143 leader_election.cc:302] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e473682ab25f42d8b91571fdda6ec179; no voters:
I20260809 19:24:55.494833 9143 leader_election.cc:288] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:55.494876 9154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:55.495034 9154 raft_consensus.cc:686] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [term 1 LEADER]: Becoming Leader. State: Replica: e473682ab25f42d8b91571fdda6ec179, State: Running, Role: LEADER
I20260809 19:24:55.495095 9143 sys_catalog.cc:564] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:55.495170 9154 consensus_queue.cc:237] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e473682ab25f42d8b91571fdda6ec179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } }
I20260809 19:24:55.495508 9154 sys_catalog.cc:455] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e473682ab25f42d8b91571fdda6ec179. Latest consensus state: current_term: 1 leader_uuid: "e473682ab25f42d8b91571fdda6ec179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e473682ab25f42d8b91571fdda6ec179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } } }
I20260809 19:24:55.495738 9154 sys_catalog.cc:458] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:55.495651 9156 sys_catalog.cc:455] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e473682ab25f42d8b91571fdda6ec179" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e473682ab25f42d8b91571fdda6ec179" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42127 } } }
I20260809 19:24:55.496131 9156 sys_catalog.cc:458] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:55.496131 9167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:55.496750 9167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:55.498257 9167 catalog_manager.cc:1347] Generated new cluster ID: a504578fa687414f91e81877c98a4f31
I20260809 19:24:55.498312 9167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:24:55 dist-test-slave-1vmg krb5kdc[9013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:24:55.502406 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:55.493324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:44725 (local address 127.0.0.1:42127)
0809 19:24:55.493484 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:55.493491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:55.493513 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:24:55.493603 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:55.493609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:55.493665 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:55.493778 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:55.494448 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:55.495762 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:55.496774 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:55.497191 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:55.497290 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:55.499628 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:55.499631 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:55.499632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:55.499640 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:55.501045 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:55.501393 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:55.501395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:55.501396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:55.501482 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:55.501684 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:55.501685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:55.501685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:55.501892 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:55.502064 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:55.502141 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:24:55.502236 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260809 19:24:55.503271 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:55.509739 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:55.504548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:40875 (local address 127.0.0.1:42127)
0809 19:24:55.504581 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:55.504582 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:55.504590 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:55.504651 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:55.504651 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:55.504660 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:55.504703 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:55.505162 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:55.505615 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:55.506246 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:55.506477 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:55.506529 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:55.507564 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:55.507565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:55.507565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:55.507571 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:55.508738 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:55.509061 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:55.509062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:55.509062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:55.509119 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:55.509305 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:55.509305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:55.509306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:55.509402 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:55.509497 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:55.509607 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 19:24:55.509662 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:24:55.520574 9167 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:55.521243 9167 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:55.524592 9167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e473682ab25f42d8b91571fdda6ec179: Generated new TSK 0
I20260809 19:24:55.524749 9167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:55.533382 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9036
W20260809 19:24:55.545650 9009 connection.cc:535] client connection to 127.0.0.1:42127 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42127): Connection reset by peer (error 104)
2026-08-09T19:24:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:55 dist-test-slave-1vmg krb5kdc[9194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:55 dist-test-slave-1vmg krb5kdc[9194](info): set up 2 sockets
Aug 09 19:24:55 dist-test-slave-1vmg krb5kdc[9194](info): commencing operation
krb5kdc: starting...
W20260809 19:24:56.654644 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:56 dist-test-slave-1vmg krb5kdc[9194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:56.769129 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41639
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:56.848768 9218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:56.848971 9218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:56.849004 9218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:56.850145 9218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:56.850193 9218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:56.850208 9218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:56.850221 9218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:56.850241 9218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:56.851667 9218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:56.851863 9218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:56.852051 9218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:56.856755 9225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:56.856683 9230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:56.856724 9218 server_base.cc:1034] running on GCE node
W20260809 19:24:56.856712 9226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:56.857139 9218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:56.857414 9218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:56.858568 9218 hybrid_clock.cc:648] HybridClock initialized: now 1786303496858554 us; error 25 us; skew 500 ppm
Aug 09 19:24:56 dist-test-slave-1vmg krb5kdc[9194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303496, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:56.861006 9218 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:56.861439 9218 webserver.cc:460] Webserver started at http://127.0.0.1:46851/ using document root <none> and password file <none>
I20260809 19:24:56.861649 9218 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:56.861701 9218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:56.861835 9218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:56.862844 9218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "5666bdc4574644d9aa2819c294c96190"
format_stamp: "Formatted at 2026-08-09 19:24:56 on dist-test-slave-1vmg"
server_key: "d836d25df1e529b8a61c291fae8e32c9"
server_key_iv: "393f3c0991717f6c30057ccdff5d4199"
server_key_version: "encryptionkey@0"
I20260809 19:24:56.863268 9218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "5666bdc4574644d9aa2819c294c96190"
format_stamp: "Formatted at 2026-08-09 19:24:56 on dist-test-slave-1vmg"
server_key: "d836d25df1e529b8a61c291fae8e32c9"
server_key_iv: "393f3c0991717f6c30057ccdff5d4199"
server_key_version: "encryptionkey@0"
I20260809 19:24:56.864535 9218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:24:56.865207 9238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:56.865365 9218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:56.865455 9218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "5666bdc4574644d9aa2819c294c96190"
format_stamp: "Formatted at 2026-08-09 19:24:56 on dist-test-slave-1vmg"
server_key: "d836d25df1e529b8a61c291fae8e32c9"
server_key_iv: "393f3c0991717f6c30057ccdff5d4199"
server_key_version: "encryptionkey@0"
I20260809 19:24:56.865527 9218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:56.887030 9218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:56.888660 9218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:56.888803 9218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:56.892983 9218 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41639
I20260809 19:24:56.893014 9314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41639 every 8 connection(s)
I20260809 19:24:56.893347 9218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:56.893712 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9218
I20260809 19:24:56.893831 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:56.894016 9581 external_mini_cluster.cc:1428] Setting key f21cf877dbcf03928c36033584a418e3
I20260809 19:24:56.894348 9316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:56.897388 9316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190: Bootstrap starting.
I20260809 19:24:56.898087 9316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:56.898406 9316 log.cc:824] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:56.899153 9316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190: No bootstrap required, opened a new log
I20260809 19:24:56.899605 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:56.894512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:57077 (local address 127.0.0.1:41639)
0809 19:24:56.894696 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:56.894703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:56.894729 (+ 26us) server_negotiation.cc:407] Connection header received
0809 19:24:56.895187 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:56.895194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:56.895245 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:56.895358 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:56.895580 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:56.896436 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:56.897521 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:56.898103 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:56.898230 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:56.898938 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:56.898943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:56.898943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:56.899098 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:56.899133 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:56.899144 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:56.899208 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:56.899385 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:24:56.899409 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260809 19:24:56.900522 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:56.901521 9316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5666bdc4574644d9aa2819c294c96190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41639 } }
I20260809 19:24:56.901705 9316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:56.901758 9316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5666bdc4574644d9aa2819c294c96190, State: Initialized, Role: FOLLOWER
I20260809 19:24:56.901880 9316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5666bdc4574644d9aa2819c294c96190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41639 } }
I20260809 19:24:56.901961 9316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:56.901988 9316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:56.902029 9316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:56.902702 9316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5666bdc4574644d9aa2819c294c96190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41639 } }
I20260809 19:24:56.902805 9316 leader_election.cc:302] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5666bdc4574644d9aa2819c294c96190; no voters:
I20260809 19:24:56.903002 9316 leader_election.cc:288] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:56.903024 9335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:56.903153 9335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [term 1 LEADER]: Becoming Leader. State: Replica: 5666bdc4574644d9aa2819c294c96190, State: Running, Role: LEADER
I20260809 19:24:56.903231 9335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5666bdc4574644d9aa2819c294c96190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41639 } }
I20260809 19:24:56.903237 9316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:56.903627 9338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5666bdc4574644d9aa2819c294c96190. Latest consensus state: current_term: 1 leader_uuid: "5666bdc4574644d9aa2819c294c96190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5666bdc4574644d9aa2819c294c96190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41639 } } }
I20260809 19:24:56.903700 9338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:56.903602 9337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5666bdc4574644d9aa2819c294c96190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5666bdc4574644d9aa2819c294c96190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41639 } } }
I20260809 19:24:56.903899 9337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:56.904287 9341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:56.904790 9341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:56.905568 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:56.902105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:35449 (local address 127.0.0.1:41639)
0809 19:24:56.902148 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:56.902149 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:56.902227 (+ 78us) server_negotiation.cc:407] Connection header received
0809 19:24:56.902255 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:56.902256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:56.902265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:56.902330 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:56.902445 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:56.903115 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:56.904132 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:56.904401 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:56.904481 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:56.905181 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:56.905182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:56.905182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:56.905187 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:56.905202 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:56.905206 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:56.905260 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:56.905508 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 19:24:56.905520 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:24:56.906288 9341 catalog_manager.cc:1347] Generated new cluster ID: 18e6c3f24c1e4388bd3eef8036743d58
I20260809 19:24:56.906344 9341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:56.924032 9341 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:56.924531 9341 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:56.932534 9341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5666bdc4574644d9aa2819c294c96190: Generated new TSK 0
I20260809 19:24:56.932717 9341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:56.941663 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9218
W20260809 19:24:56.956465 9190 connection.cc:535] client connection to 127.0.0.1:41639 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41639): Connection reset by peer (error 104)
2026-08-09T19:24:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:57 dist-test-slave-1vmg krb5kdc[9365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:57 dist-test-slave-1vmg krb5kdc[9365](info): set up 2 sockets
Aug 09 19:24:57 dist-test-slave-1vmg krb5kdc[9365](info): commencing operation
krb5kdc: starting...
W20260809 19:24:58.089668 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:58 dist-test-slave-1vmg krb5kdc[9365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:58.198541 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36277
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:58.276878 9387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:58.277082 9387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:58.277119 9387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:58.278247 9387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:58.278293 9387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:58.278316 9387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:58.278335 9387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:58.278366 9387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:58.279883 9387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:58.280128 9387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:58.280323 9387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:58.284833 9405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:58.284839 9399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:58.284832 9398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:58.284916 9387 server_base.cc:1034] running on GCE node
I20260809 19:24:58.285243 9387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:58.285502 9387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:58.286636 9387 hybrid_clock.cc:648] HybridClock initialized: now 1786303498286627 us; error 27 us; skew 500 ppm
Aug 09 19:24:58 dist-test-slave-1vmg krb5kdc[9365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303498, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:58.289006 9387 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:58.289407 9387 webserver.cc:460] Webserver started at http://127.0.0.1:35393/ using document root <none> and password file <none>
I20260809 19:24:58.289609 9387 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:58.289651 9387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:58.289746 9387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:58.290668 9387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "d2a5003babb44018807904ab1f2baca0"
format_stamp: "Formatted at 2026-08-09 19:24:58 on dist-test-slave-1vmg"
server_key: "f6f4d71971d8765eb1513665b9397001"
server_key_iv: "f3b2cbb3d0fcd6db77b3cbacfe0f23a4"
server_key_version: "encryptionkey@0"
I20260809 19:24:58.291006 9387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "d2a5003babb44018807904ab1f2baca0"
format_stamp: "Formatted at 2026-08-09 19:24:58 on dist-test-slave-1vmg"
server_key: "f6f4d71971d8765eb1513665b9397001"
server_key_iv: "f3b2cbb3d0fcd6db77b3cbacfe0f23a4"
server_key_version: "encryptionkey@0"
I20260809 19:24:58.292238 9387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:24:58.292881 9415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:58.293018 9387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:24:58.293102 9387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "d2a5003babb44018807904ab1f2baca0"
format_stamp: "Formatted at 2026-08-09 19:24:58 on dist-test-slave-1vmg"
server_key: "f6f4d71971d8765eb1513665b9397001"
server_key_iv: "f3b2cbb3d0fcd6db77b3cbacfe0f23a4"
server_key_version: "encryptionkey@0"
I20260809 19:24:58.293172 9387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:58.296379 9387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:58.298012 9387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:58.298141 9387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:58.302078 9387 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36277
I20260809 19:24:58.302124 9495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36277 every 8 connection(s)
I20260809 19:24:58.302392 9387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:58.303328 9496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:58.305330 9496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0: Bootstrap starting.
I20260809 19:24:58.305855 9496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:58.306105 9496 log.cc:824] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:58.306730 9496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0: No bootstrap required, opened a new log
I20260809 19:24:58.308280 9496 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a5003babb44018807904ab1f2baca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36277 } }
I20260809 19:24:58.308403 9496 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:58.308424 9496 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2a5003babb44018807904ab1f2baca0, State: Initialized, Role: FOLLOWER
I20260809 19:24:58.308552 9496 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a5003babb44018807904ab1f2baca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36277 } }
I20260809 19:24:58.308614 9496 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:58.308636 9496 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:58.308682 9496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:58.309149 9496 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a5003babb44018807904ab1f2baca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36277 } }
I20260809 19:24:58.309237 9496 leader_election.cc:302] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2a5003babb44018807904ab1f2baca0; no voters:
I20260809 19:24:58.309379 9496 leader_election.cc:288] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:58.309417 9502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:58.309559 9502 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [term 1 LEADER]: Becoming Leader. State: Replica: d2a5003babb44018807904ab1f2baca0, State: Running, Role: LEADER
I20260809 19:24:58.309603 9496 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:58.309649 9502 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a5003babb44018807904ab1f2baca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36277 } }
I20260809 19:24:58.310003 9505 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2a5003babb44018807904ab1f2baca0. Latest consensus state: current_term: 1 leader_uuid: "d2a5003babb44018807904ab1f2baca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a5003babb44018807904ab1f2baca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36277 } } }
I20260809 19:24:58.310103 9503 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2a5003babb44018807904ab1f2baca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a5003babb44018807904ab1f2baca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36277 } } }
I20260809 19:24:58.310168 9505 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:58.310227 9503 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:58.310578 9516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:58.311074 9516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:58.312371 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9387
I20260809 19:24:58.312505 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:58.312492 9516 catalog_manager.cc:1347] Generated new cluster ID: 43796a9ed63348c98470c276127c0458
I20260809 19:24:58.312534 9516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:24:58.312656 9581 external_mini_cluster.cc:1428] Setting key dcdefd335bf25c749b7b1c4f93135a2b
I20260809 19:24:58.320338 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:58.313171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:40825 (local address 127.0.0.1:36277)
0809 19:24:58.319021 (+ 5850us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:58.319029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:58.319056 (+ 27us) server_negotiation.cc:407] Connection header received
0809 19:24:58.319139 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:58.319144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:58.319165 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:58.319279 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:58.319932 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:58.319935 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:58.319935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:58.319939 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:58.319979 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:58.319989 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:58.320046 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:58.320123 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:24:58.320166 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5808,"thread_start_us":74,"threads_started":1}
I20260809 19:24:58.321256 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:58.321377 9516 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:58.321508 9516 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:58.323659 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:58.322812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:40867 (local address 127.0.0.1:36277)
0809 19:24:58.322847 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:58.322849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:58.322857 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:24:58.322946 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:58.322946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:58.322950 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:58.322997 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:58.323486 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:58.323487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:58.323487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:24:58.323491 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:58.323502 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:24:58.323506 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:58.323543 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:58.323591 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 19:24:58.323603 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:24:58.333551 9516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2a5003babb44018807904ab1f2baca0: Generated new TSK 0
I20260809 19:24:58.333698 9516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:58.337283 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9387
2026-08-09T19:24:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:58 dist-test-slave-1vmg krb5kdc[9554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:58 dist-test-slave-1vmg krb5kdc[9554](info): set up 2 sockets
Aug 09 19:24:58 dist-test-slave-1vmg krb5kdc[9554](info): commencing operation
krb5kdc: starting...
W20260809 19:24:59.453102 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:24:59 dist-test-slave-1vmg krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:24:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:24:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:24:59.559615 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:24:59.636965 9574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:24:59.637147 9574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:24:59.637168 9574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:24:59.638279 9574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:24:59.638319 9574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:24:59.638335 9574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:24:59.638360 9574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:24:59.638382 9574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:24:59.639820 9574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41099
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:24:59.640027 9574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:24:59.640219 9574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:24:59.644513 9589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:59.644521 9594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:24:59.644524 9592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:24:59.644761 9574 server_base.cc:1034] running on GCE node
I20260809 19:24:59.644980 9574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:24:59.645253 9574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:24:59.646399 9574 hybrid_clock.cc:648] HybridClock initialized: now 1786303499646386 us; error 28 us; skew 500 ppm
Aug 09 19:24:59 dist-test-slave-1vmg krb5kdc[9554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303499, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:24:59.648587 9574 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:24:59.648973 9574 webserver.cc:460] Webserver started at http://127.0.0.1:32929/ using document root <none> and password file <none>
I20260809 19:24:59.649163 9574 fs_manager.cc:359] Metadata directory not provided
I20260809 19:24:59.649209 9574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:24:59.649327 9574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:24:59.650231 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "a54c1ba84654470a9b19fa6147258678"
format_stamp: "Formatted at 2026-08-09 19:24:59 on dist-test-slave-1vmg"
server_key: "a13a7fdfdc0e7355dc6249f96ea6ff48"
server_key_iv: "a9909f8b30c9f2b10f65677b149dd5d5"
server_key_version: "encryptionkey@0"
I20260809 19:24:59.650533 9574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "a54c1ba84654470a9b19fa6147258678"
format_stamp: "Formatted at 2026-08-09 19:24:59 on dist-test-slave-1vmg"
server_key: "a13a7fdfdc0e7355dc6249f96ea6ff48"
server_key_iv: "a9909f8b30c9f2b10f65677b149dd5d5"
server_key_version: "encryptionkey@0"
I20260809 19:24:59.651783 9574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:24:59.652585 9606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:24:59.652756 9574 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:24:59.652830 9574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "a54c1ba84654470a9b19fa6147258678"
format_stamp: "Formatted at 2026-08-09 19:24:59 on dist-test-slave-1vmg"
server_key: "a13a7fdfdc0e7355dc6249f96ea6ff48"
server_key_iv: "a9909f8b30c9f2b10f65677b149dd5d5"
server_key_version: "encryptionkey@0"
I20260809 19:24:59.652879 9574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:24:59.667665 9574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:24:59.669517 9574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:24:59.669624 9574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:24:59.673266 9574 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41099
I20260809 19:24:59.673301 9689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41099 every 8 connection(s)
I20260809 19:24:59.673604 9574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:24:59.674477 9690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:24:59.676517 9690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678: Bootstrap starting.
I20260809 19:24:59.677038 9690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678: Neither blocks nor log segments found. Creating new log.
I20260809 19:24:59.677294 9690 log.cc:824] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678: Log is configured to *not* fsync() on all Append() calls
I20260809 19:24:59.677891 9690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678: No bootstrap required, opened a new log
I20260809 19:24:59.679432 9690 raft_consensus.cc:348] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c1ba84654470a9b19fa6147258678" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260809 19:24:59.679553 9690 raft_consensus.cc:374] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:24:59.679587 9690 raft_consensus.cc:729] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a54c1ba84654470a9b19fa6147258678, State: Initialized, Role: FOLLOWER
I20260809 19:24:59.679729 9690 consensus_queue.cc:260] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c1ba84654470a9b19fa6147258678" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260809 19:24:59.679801 9690 raft_consensus.cc:388] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:24:59.679826 9690 raft_consensus.cc:482] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:24:59.679869 9690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:24:59.680344 9690 raft_consensus.cc:504] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c1ba84654470a9b19fa6147258678" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260809 19:24:59.680433 9690 leader_election.cc:302] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a54c1ba84654470a9b19fa6147258678; no voters:
I20260809 19:24:59.680575 9690 leader_election.cc:288] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:24:59.680610 9694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:24:59.680733 9694 raft_consensus.cc:686] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [term 1 LEADER]: Becoming Leader. State: Replica: a54c1ba84654470a9b19fa6147258678, State: Running, Role: LEADER
I20260809 19:24:59.680820 9690 sys_catalog.cc:564] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:24:59.680889 9694 consensus_queue.cc:237] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c1ba84654470a9b19fa6147258678" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } }
I20260809 19:24:59.681342 9695 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a54c1ba84654470a9b19fa6147258678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c1ba84654470a9b19fa6147258678" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } } }
I20260809 19:24:59.681473 9696 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a54c1ba84654470a9b19fa6147258678. Latest consensus state: current_term: 1 leader_uuid: "a54c1ba84654470a9b19fa6147258678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54c1ba84654470a9b19fa6147258678" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41099 } } }
I20260809 19:24:59.681586 9696 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:59.681529 9695 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678 [sys.catalog]: This master's current role is: LEADER
I20260809 19:24:59.682009 9708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:24:59.682500 9708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:24:59.683554 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9574
I20260809 19:24:59.683686 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:24:59.683849 9581 external_mini_cluster.cc:1428] Setting key 8b1055f5f624597ff64863d3448cd562
I20260809 19:24:59.683921 9708 catalog_manager.cc:1347] Generated new cluster ID: 046790d520694b989bdbab476ff23e6c
I20260809 19:24:59.683964 9708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:24:59 dist-test-slave-1vmg krb5kdc[9554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 19:24:59.693805 9708 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:24:59.694446 9708 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:24:59.695196 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:59.684351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:36359 (local address 127.0.0.1:41099)
0809 19:24:59.687084 (+ 2733us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:59.687092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:59.687114 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:24:59.687199 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:59.687204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:59.687250 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:59.687358 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:59.687958 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:59.688841 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:59.689550 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:59.689944 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:59.690034 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:59.692145 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:59.692148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:59.692148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:59.692158 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:59.693628 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:59.694060 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:59.694063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:59.694063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:59.694148 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:59.694347 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:59.694348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:59.694348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:59.694567 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:59.694722 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:59.694856 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 19:24:59.694973 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2703,"thread_start_us":72,"threads_started":1}
I20260809 19:24:59.696053 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:24:59.702602 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:24:59.697264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:38561 (local address 127.0.0.1:41099)
0809 19:24:59.697298 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:24:59.697300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:24:59.697408 (+ 108us) server_negotiation.cc:407] Connection header received
0809 19:24:59.697453 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:24:59.697454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:24:59.697462 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:24:59.697513 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:24:59.697984 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:59.698461 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:59.699187 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:24:59.699394 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:24:59.699444 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:24:59.700460 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:24:59.700461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:24:59.700461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:24:59.700468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:24:59.701618 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:59.701938 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:59.701939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:59.701939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:59.701990 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:24:59.702174 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:24:59.702174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:24:59.702175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:24:59.702271 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:24:59.702355 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 19:24:59.702471 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 19:24:59.702526 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:24:59.705338 9708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a54c1ba84654470a9b19fa6147258678: Generated new TSK 0
I20260809 19:24:59.705490 9708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:24:59.712074 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9574
W20260809 19:24:59.722184 9551 connection.cc:535] client connection to 127.0.0.1:41099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41099): Connection reset by peer (error 104)
2026-08-09T19:24:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:24:59 dist-test-slave-1vmg krb5kdc[9733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:24:59 dist-test-slave-1vmg krb5kdc[9733](info): set up 2 sockets
Aug 09 19:24:59 dist-test-slave-1vmg krb5kdc[9733](info): commencing operation
krb5kdc: starting...
W20260809 19:25:00.859926 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:00 dist-test-slave-1vmg krb5kdc[9733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:25:00.966243 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46839
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:01.046216 9753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:01.046397 9753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:01.046419 9753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:01.047487 9753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:01.047518 9753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:01.047533 9753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:01.047546 9753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:01.047560 9753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:01.048933 9753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:01.049113 9753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:01.049274 9753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:01.053634 9764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:01.053653 9760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:01.053674 9759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:01.053783 9753 server_base.cc:1034] running on GCE node
I20260809 19:25:01.054083 9753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:01.054337 9753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:01.055471 9753 hybrid_clock.cc:648] HybridClock initialized: now 1786303501055456 us; error 26 us; skew 500 ppm
Aug 09 19:25:01 dist-test-slave-1vmg krb5kdc[9733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:01.062419 9753 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:25:01.062779 9753 webserver.cc:460] Webserver started at http://127.0.0.1:37319/ using document root <none> and password file <none>
I20260809 19:25:01.062988 9753 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:01.063032 9753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:01.063123 9753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:01.064024 9753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "997fa07b860a45bd86a9e166c76da6bb"
format_stamp: "Formatted at 2026-08-09 19:25:01 on dist-test-slave-1vmg"
server_key: "986cb49c3e636d6a4615943b603fa489"
server_key_iv: "f434ae4c600ecad709b8112248758e57"
server_key_version: "encryptionkey@0"
I20260809 19:25:01.064301 9753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "997fa07b860a45bd86a9e166c76da6bb"
format_stamp: "Formatted at 2026-08-09 19:25:01 on dist-test-slave-1vmg"
server_key: "986cb49c3e636d6a4615943b603fa489"
server_key_iv: "f434ae4c600ecad709b8112248758e57"
server_key_version: "encryptionkey@0"
I20260809 19:25:01.065527 9753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:01.066151 9772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:01.066320 9753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:01.066413 9753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "997fa07b860a45bd86a9e166c76da6bb"
format_stamp: "Formatted at 2026-08-09 19:25:01 on dist-test-slave-1vmg"
server_key: "986cb49c3e636d6a4615943b603fa489"
server_key_iv: "f434ae4c600ecad709b8112248758e57"
server_key_version: "encryptionkey@0"
I20260809 19:25:01.066473 9753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:01.086329 9753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:01.088376 9753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:01.088498 9753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:01.092391 9753 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46839
I20260809 19:25:01.092422 9846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46839 every 8 connection(s)
I20260809 19:25:01.092733 9753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:01.093533 9847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:01.095566 9847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb: Bootstrap starting.
I20260809 19:25:01.096086 9847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:01.096359 9847 log.cc:824] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:01.096982 9847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb: No bootstrap required, opened a new log
I20260809 19:25:01.098485 9847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997fa07b860a45bd86a9e166c76da6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46839 } }
I20260809 19:25:01.098606 9847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:01.098639 9847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997fa07b860a45bd86a9e166c76da6bb, State: Initialized, Role: FOLLOWER
I20260809 19:25:01.098764 9847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997fa07b860a45bd86a9e166c76da6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46839 } }
I20260809 19:25:01.098841 9847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:01.098872 9847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:01.098938 9847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:01.099413 9847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997fa07b860a45bd86a9e166c76da6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46839 } }
I20260809 19:25:01.099503 9847 leader_election.cc:302] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 997fa07b860a45bd86a9e166c76da6bb; no voters:
I20260809 19:25:01.099645 9847 leader_election.cc:288] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:01.099680 9851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:01.099852 9851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [term 1 LEADER]: Becoming Leader. State: Replica: 997fa07b860a45bd86a9e166c76da6bb, State: Running, Role: LEADER
I20260809 19:25:01.099884 9847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:01.099947 9851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997fa07b860a45bd86a9e166c76da6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46839 } }
I20260809 19:25:01.100237 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9753
I20260809 19:25:01.100355 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:01.100438 9855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "997fa07b860a45bd86a9e166c76da6bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997fa07b860a45bd86a9e166c76da6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46839 } } }
I20260809 19:25:01.100517 9855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:01.100555 9581 external_mini_cluster.cc:1428] Setting key b2469eb6144947406c3fbe114a158ea3
I20260809 19:25:01.101219 9856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 997fa07b860a45bd86a9e166c76da6bb. Latest consensus state: current_term: 1 leader_uuid: "997fa07b860a45bd86a9e166c76da6bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "997fa07b860a45bd86a9e166c76da6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46839 } } }
I20260809 19:25:01.101418 9856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb [sys.catalog]: This master's current role is: LEADER
W20260809 19:25:01.101682 9878 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:01.101727 9878 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:01.101780 9868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:01.101976 9868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:01.103416 9868 catalog_manager.cc:1347] Generated new cluster ID: 9986f35435754e49ab5a8511dee8e2ab
I20260809 19:25:01.103475 9868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:01.106508 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:01.101303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:60837 (local address 127.0.0.1:46839)
0809 19:25:01.101599 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:01.101608 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:01.101640 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:25:01.101749 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:01.101756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:01.102615 (+ 859us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:01.102766 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:01.103024 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:01.104007 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:01.104790 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:01.105203 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:01.105294 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:01.105899 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:01.105903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:01.105904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:01.106053 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:01.106080 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:01.106103 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:01.106170 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:01.106274 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 19:25:01.106302 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":82,"threads_started":1}
I20260809 19:25:01.107393 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:01.113440 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:01.108603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:53815 (local address 127.0.0.1:46839)
0809 19:25:01.108637 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:01.108638 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:01.111060 (+ 2422us) server_negotiation.cc:407] Connection header received
0809 19:25:01.111126 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:01.111127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:01.111138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:01.111191 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:01.111314 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:01.111769 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:01.112435 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:01.112639 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:01.112690 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:01.113223 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:01.113224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:01.113225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:01.113229 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:01.113240 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:01.113244 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:01.113292 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:01.113361 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 19:25:01.113371 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:25:01.114717 9868 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:01.115253 9868 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:01.120971 9868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 997fa07b860a45bd86a9e166c76da6bb: Generated new TSK 0
I20260809 19:25:01.121121 9868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:01.126639 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9753
W20260809 19:25:01.136914 9730 connection.cc:535] client connection to 127.0.0.1:46839 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46839): Connection reset by peer (error 104)
2026-08-09T19:25:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:01 dist-test-slave-1vmg krb5kdc[9894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:01 dist-test-slave-1vmg krb5kdc[9894](info): set up 2 sockets
Aug 09 19:25:01 dist-test-slave-1vmg krb5kdc[9894](info): commencing operation
krb5kdc: starting...
W20260809 19:25:02.276022 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:02 dist-test-slave-1vmg krb5kdc[9894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 19:25:02.383067 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:02.461956 9916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:02.462145 9916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:02.462165 9916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:02.463274 9916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:02.463311 9916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:02.463325 9916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:02.463348 9916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:02.463370 9916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:02.464810 9916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:02.465030 9916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:02.465229 9916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:02.469642 9925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:02.469652 9929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:02.469655 9926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:02.469705 9916 server_base.cc:1034] running on GCE node
I20260809 19:25:02.470026 9916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:02.470306 9916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:02.471480 9916 hybrid_clock.cc:648] HybridClock initialized: now 1786303502471459 us; error 35 us; skew 500 ppm
Aug 09 19:25:02 dist-test-slave-1vmg krb5kdc[9894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303502, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:02.476655 9916 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 19:25:02.477066 9916 webserver.cc:460] Webserver started at http://127.0.0.1:43293/ using document root <none> and password file <none>
I20260809 19:25:02.477278 9916 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:02.477330 9916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:02.477464 9916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:02.478457 9916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "8b39d4fa4ce444169642a68b2c1cabfc"
format_stamp: "Formatted at 2026-08-09 19:25:02 on dist-test-slave-1vmg"
server_key: "71d052230a92f0798d5245e802e816fb"
server_key_iv: "5113d23de37396fbe203b0ab244677fe"
server_key_version: "encryptionkey@0"
I20260809 19:25:02.478812 9916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "8b39d4fa4ce444169642a68b2c1cabfc"
format_stamp: "Formatted at 2026-08-09 19:25:02 on dist-test-slave-1vmg"
server_key: "71d052230a92f0798d5245e802e816fb"
server_key_iv: "5113d23de37396fbe203b0ab244677fe"
server_key_version: "encryptionkey@0"
I20260809 19:25:02.480139 9916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:02.480855 9938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:02.481036 9916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:02.481127 9916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "8b39d4fa4ce444169642a68b2c1cabfc"
format_stamp: "Formatted at 2026-08-09 19:25:02 on dist-test-slave-1vmg"
server_key: "71d052230a92f0798d5245e802e816fb"
server_key_iv: "5113d23de37396fbe203b0ab244677fe"
server_key_version: "encryptionkey@0"
I20260809 19:25:02.481189 9916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:02.484382 9916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:02.485929 9916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:02.486050 9916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:02.489753 9916 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34889
I20260809 19:25:02.489782 10007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34889 every 8 connection(s)
I20260809 19:25:02.490059 9916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:02.490952 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:02.492966 10008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc: Bootstrap starting.
I20260809 19:25:02.493482 10008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:02.493736 10008 log.cc:824] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:02.494410 10008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc: No bootstrap required, opened a new log
I20260809 19:25:02.495958 10008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34889 } }
I20260809 19:25:02.496086 10008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:02.496106 10008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b39d4fa4ce444169642a68b2c1cabfc, State: Initialized, Role: FOLLOWER
I20260809 19:25:02.496222 10008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34889 } }
I20260809 19:25:02.496281 10008 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:02.496300 10008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:02.496330 10008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:02.496778 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 9916
I20260809 19:25:02.496825 10008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34889 } }
I20260809 19:25:02.496888 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:02.496923 10008 leader_election.cc:302] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b39d4fa4ce444169642a68b2c1cabfc; no voters:
I20260809 19:25:02.497056 9581 external_mini_cluster.cc:1428] Setting key 5bfa780920b8da53a7786fc228c23cd1
I20260809 19:25:02.497076 10008 leader_election.cc:288] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:02.497115 10013 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:02.497257 10013 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [term 1 LEADER]: Becoming Leader. State: Replica: 8b39d4fa4ce444169642a68b2c1cabfc, State: Running, Role: LEADER
I20260809 19:25:02.497377 10008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:02.497390 10013 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34889 } }
I20260809 19:25:02.497742 10013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b39d4fa4ce444169642a68b2c1cabfc. Latest consensus state: current_term: 1 leader_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34889 } } }
I20260809 19:25:02.497938 10013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:02.497884 10014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b39d4fa4ce444169642a68b2c1cabfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34889 } } }
I20260809 19:25:02.498366 10014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:02.498533 10021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:02.499053 10021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:02.499527 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:02.497613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:32817 (local address 127.0.0.1:34889)
0809 19:25:02.497980 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:02.497989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:02.498017 (+ 28us) server_negotiation.cc:407] Connection header received
0809 19:25:02.498140 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:02.498147 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:02.498177 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:02.498568 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:02.499034 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:02.499037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:02.499037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:02.499040 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:02.499074 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:02.499088 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:02.499304 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:02.499334 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 19:25:02.499365 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":68,"threads_started":1}
I20260809 19:25:02.500399 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:02.502681 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:02.501875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:58347 (local address 127.0.0.1:34889)
0809 19:25:02.501912 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:02.501913 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:02.501962 (+ 49us) server_negotiation.cc:407] Connection header received
0809 19:25:02.502035 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:02.502036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:02.502039 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:02.502079 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:02.502514 (+ 435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:02.502515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:02.502515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:02.502518 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:02.502528 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:02.502531 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:02.502569 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:02.502620 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 19:25:02.502629 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:25:02.503419 10021 catalog_manager.cc:1347] Generated new cluster ID: 9e8ffc7cb46c428f864b973562c4b1ba
I20260809 19:25:02.503472 10021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:02.520641 10021 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:02.520807 10021 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:02.528162 10021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b39d4fa4ce444169642a68b2c1cabfc: Generated new TSK 0
I20260809 19:25:02.528342 10021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:02.536586 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 9916
2026-08-09T19:25:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:02 dist-test-slave-1vmg krb5kdc[10058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:02 dist-test-slave-1vmg krb5kdc[10058](info): set up 2 sockets
Aug 09 19:25:02 dist-test-slave-1vmg krb5kdc[10058](info): commencing operation
krb5kdc: starting...
W20260809 19:25:03.662184 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:03 dist-test-slave-1vmg krb5kdc[10058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:03.769115 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:45445
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:45905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:45445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:03.850047 10075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:03.850224 10075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:03.850243 10075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:03.851351 10075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:03.851387 10075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:03.851400 10075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:03.851423 10075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:03.851435 10075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:03.852813 10075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:45905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:45445
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:45445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:03.852984 10075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:03.853154 10075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:03.857491 10088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:03.857621 10075 server_base.cc:1034] running on GCE node
W20260809 19:25:03.857494 10090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:03.857556 10084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:03.857921 10075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:03.858151 10075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:03.859273 10075 hybrid_clock.cc:648] HybridClock initialized: now 1786303503859264 us; error 28 us; skew 500 ppm
Aug 09 19:25:03 dist-test-slave-1vmg krb5kdc[10058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303503, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:03.861570 10075 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:03.861958 10075 webserver.cc:460] Webserver started at http://127.9.91.126:33803/ using document root <none> and password file <none>
I20260809 19:25:03.862164 10075 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:03.862205 10075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:03.862313 10075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:03.863302 10075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "60077f9b4abb4b76a0967e7b4429bc49"
format_stamp: "Formatted at 2026-08-09 19:25:03 on dist-test-slave-1vmg"
server_key: "60433e21aff3c4c37dbfa9a3a9acc1c3"
server_key_iv: "990577d5db15b76c583bb8ea5d5530e5"
server_key_version: "encryptionkey@0"
I20260809 19:25:03.863610 10075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "60077f9b4abb4b76a0967e7b4429bc49"
format_stamp: "Formatted at 2026-08-09 19:25:03 on dist-test-slave-1vmg"
server_key: "60433e21aff3c4c37dbfa9a3a9acc1c3"
server_key_iv: "990577d5db15b76c583bb8ea5d5530e5"
server_key_version: "encryptionkey@0"
I20260809 19:25:03.864807 10075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:03.865417 10101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:03.865561 10075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:03.865648 10075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "60077f9b4abb4b76a0967e7b4429bc49"
format_stamp: "Formatted at 2026-08-09 19:25:03 on dist-test-slave-1vmg"
server_key: "60433e21aff3c4c37dbfa9a3a9acc1c3"
server_key_iv: "990577d5db15b76c583bb8ea5d5530e5"
server_key_version: "encryptionkey@0"
I20260809 19:25:03.865706 10075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:03.876128 10075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:03.877986 10075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:03.878111 10075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:03.881677 10075 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:45445
I20260809 19:25:03.881713 10175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:45445 every 8 connection(s)
I20260809 19:25:03.881985 10075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:03.882735 10176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:03.884048 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10075
I20260809 19:25:03.884166 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:03.884366 9581 external_mini_cluster.cc:1428] Setting key 4a69140b85d9eee9579583898386ebe9
I20260809 19:25:03.885082 10176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49: Bootstrap starting.
I20260809 19:25:03.885638 10176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:03.885967 10176 log.cc:824] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:03.886624 10176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49: No bootstrap required, opened a new log
I20260809 19:25:03.888650 10176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60077f9b4abb4b76a0967e7b4429bc49" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 45445 } }
I20260809 19:25:03.888773 10176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:03.888796 10176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60077f9b4abb4b76a0967e7b4429bc49, State: Initialized, Role: FOLLOWER
I20260809 19:25:03.888945 10176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60077f9b4abb4b76a0967e7b4429bc49" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 45445 } }
I20260809 19:25:03.889034 10176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:03.889091 10176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:03.889130 10176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:03.889719 10176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60077f9b4abb4b76a0967e7b4429bc49" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 45445 } }
I20260809 19:25:03.889835 10176 leader_election.cc:302] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60077f9b4abb4b76a0967e7b4429bc49; no voters:
I20260809 19:25:03.889997 10176 leader_election.cc:288] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:03.890025 10183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:03.890180 10183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [term 1 LEADER]: Becoming Leader. State: Replica: 60077f9b4abb4b76a0967e7b4429bc49, State: Running, Role: LEADER
I20260809 19:25:03.890241 10176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:03.890332 10183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60077f9b4abb4b76a0967e7b4429bc49" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 45445 } }
I20260809 19:25:03.890704 10185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60077f9b4abb4b76a0967e7b4429bc49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60077f9b4abb4b76a0967e7b4429bc49" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 45445 } } }
I20260809 19:25:03.890779 10185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [sys.catalog]: This master's current role is: LEADER
Aug 09 19:25:03 dist-test-slave-1vmg krb5kdc[10058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:03.891033 10183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60077f9b4abb4b76a0967e7b4429bc49. Latest consensus state: current_term: 1 leader_uuid: "60077f9b4abb4b76a0967e7b4429bc49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60077f9b4abb4b76a0967e7b4429bc49" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 45445 } } }
I20260809 19:25:03.891299 10183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:03.891439 10191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:03.892167 10191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:03.893769 10191 catalog_manager.cc:1347] Generated new cluster ID: 2ddf6303057547aa967e19f2882ca66f
I20260809 19:25:03.893818 10191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:03.894935 10181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:03.884840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42307 (local address 127.9.91.126:45445)
0809 19:25:03.884979 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:03.884987 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:03.885009 (+ 22us) server_negotiation.cc:407] Connection header received
0809 19:25:03.885401 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:03.885407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:03.885451 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:03.885554 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:03.886351 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:03.887332 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:03.888397 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:03.888761 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:03.888868 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:03.891447 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:03.891451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:03.891451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:03.891462 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:03.893248 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:03.893679 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:03.893682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:03.893683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:03.893786 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:03.894035 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:03.894036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:03.894036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:03.894313 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:03.894451 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:03.894622 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 19:25:03.894712 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260809 19:25:03.895823 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:03.905649 10181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:03.899170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45311 (local address 127.9.91.126:45445)
0809 19:25:03.899210 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:03.899211 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:03.899303 (+ 92us) server_negotiation.cc:407] Connection header received
0809 19:25:03.899344 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:03.899345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:03.899354 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:03.899418 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:03.899960 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:03.900695 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:03.901596 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:03.901927 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:03.901996 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:03.903186 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:03.903188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:03.903188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:03.903195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:03.904590 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:03.904951 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:03.904952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:03.904952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:03.905006 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:03.905188 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:03.905189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:03.905189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:03.905308 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:03.905401 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:03.905495 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 19:25:03.905565 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:25:03.918138 10191 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:03.918798 10191 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:03.925091 10191 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60077f9b4abb4b76a0967e7b4429bc49: Generated new TSK 0
I20260809 19:25:03.925242 10191 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:03.928067 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10075
W20260809 19:25:03.943416 10053 connection.cc:535] client connection to 127.9.91.126:45445 recv error: Network error: failed to read from TLS socket (remote: 127.9.91.126:45445): Connection reset by peer (error 104)
2026-08-09T19:25:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:03 dist-test-slave-1vmg krb5kdc[10221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:03 dist-test-slave-1vmg krb5kdc[10221](info): set up 2 sockets
Aug 09 19:25:03 dist-test-slave-1vmg krb5kdc[10221](info): commencing operation
krb5kdc: starting...
W20260809 19:25:05.082520 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:05 dist-test-slave-1vmg krb5kdc[10221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:05.190173 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:33453
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:33453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:05.268728 10241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:05.268906 10241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:05.268926 10241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:05.270009 10241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:05.270058 10241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:05.270072 10241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:05.270097 10241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:05.270123 10241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:05.271566 10241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:33453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:33453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:05.271780 10241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:05.271971 10241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:05.276849 10247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:05.276895 10248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:05.276980 10241 server_base.cc:1034] running on GCE node
W20260809 19:25:05.276850 10252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:05.277238 10241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:05.277501 10241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:05.278640 10241 hybrid_clock.cc:648] HybridClock initialized: now 1786303505278628 us; error 33 us; skew 500 ppm
Aug 09 19:25:05 dist-test-slave-1vmg krb5kdc[10221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303505, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:05.280962 10241 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:05.281383 10241 webserver.cc:460] Webserver started at http://127.9.91.126:34049/ using document root <none> and password file <none>
I20260809 19:25:05.281594 10241 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:05.281642 10241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:05.281760 10241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:05.282684 10241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "2d0df38750db4794b123c41da4a6b3ca"
format_stamp: "Formatted at 2026-08-09 19:25:05 on dist-test-slave-1vmg"
server_key: "26a9699465af82d9f1e97ca1cb29464f"
server_key_iv: "51dd93e80dd74c4268765c6584140412"
server_key_version: "encryptionkey@0"
I20260809 19:25:05.283039 10241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "2d0df38750db4794b123c41da4a6b3ca"
format_stamp: "Formatted at 2026-08-09 19:25:05 on dist-test-slave-1vmg"
server_key: "26a9699465af82d9f1e97ca1cb29464f"
server_key_iv: "51dd93e80dd74c4268765c6584140412"
server_key_version: "encryptionkey@0"
I20260809 19:25:05.284274 10241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 19:25:05.284916 10261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:05.285087 10241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:05.285158 10241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "2d0df38750db4794b123c41da4a6b3ca"
format_stamp: "Formatted at 2026-08-09 19:25:05 on dist-test-slave-1vmg"
server_key: "26a9699465af82d9f1e97ca1cb29464f"
server_key_iv: "51dd93e80dd74c4268765c6584140412"
server_key_version: "encryptionkey@0"
I20260809 19:25:05.285224 10241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:05.306056 10241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:05.307742 10241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:05.307876 10241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:05.312052 10241 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:33453
I20260809 19:25:05.312096 10352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:33453 every 8 connection(s)
I20260809 19:25:05.312397 10241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:05.313207 10353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:05.314145 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10241
I20260809 19:25:05.314257 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:05.314445 9581 external_mini_cluster.cc:1428] Setting key 0c8343be4f85a8f3dbc3568be1036c65
I20260809 19:25:05.316020 10353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca: Bootstrap starting.
I20260809 19:25:05.316723 10353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:05.317065 10353 log.cc:824] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:05.317768 10353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca: No bootstrap required, opened a new log
I20260809 19:25:05.320116 10353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df38750db4794b123c41da4a6b3ca" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 33453 } }
I20260809 19:25:05.320261 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:05.314909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38801 (local address 127.9.91.126:33453)
0809 19:25:05.315129 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:05.315138 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:05.315172 (+ 34us) server_negotiation.cc:407] Connection header received
0809 19:25:05.315704 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:05.315713 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:05.315775 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:05.315930 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:05.316074 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:05.317381 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:05.318174 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:05.318773 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:05.318902 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:05.319531 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:05.319536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:05.319537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:05.319769 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:05.319813 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:05.319829 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:05.319925 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:05.319985 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 19:25:05.320016 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260809 19:25:05.320302 10353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:05.320340 10353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0df38750db4794b123c41da4a6b3ca, State: Initialized, Role: FOLLOWER
I20260809 19:25:05.320457 10353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df38750db4794b123c41da4a6b3ca" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 33453 } }
I20260809 19:25:05.320549 10353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:05.320593 10353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:05.320636 10353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:05.321148 10353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df38750db4794b123c41da4a6b3ca" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 33453 } }
I20260809 19:25:05.321161 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:05.321251 10353 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d0df38750db4794b123c41da4a6b3ca; no voters:
I20260809 19:25:05.321412 10353 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:05.321451 10359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:05.321630 10359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [term 1 LEADER]: Becoming Leader. State: Replica: 2d0df38750db4794b123c41da4a6b3ca, State: Running, Role: LEADER
I20260809 19:25:05.321664 10353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:05.321758 10359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df38750db4794b123c41da4a6b3ca" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 33453 } }
I20260809 19:25:05.322191 10359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0df38750db4794b123c41da4a6b3ca. Latest consensus state: current_term: 1 leader_uuid: "2d0df38750db4794b123c41da4a6b3ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df38750db4794b123c41da4a6b3ca" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 33453 } } }
I20260809 19:25:05.322283 10359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:05.322446 10360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d0df38750db4794b123c41da4a6b3ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0df38750db4794b123c41da4a6b3ca" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 33453 } } }
I20260809 19:25:05.322510 10360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:05.323048 10372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:05.323598 10372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:05.325680 10372 catalog_manager.cc:1347] Generated new cluster ID: ff2db02c315a4a5b9e8248f493cb5709
I20260809 19:25:05.325734 10372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:05.326967 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:05.322588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57579 (local address 127.9.91.126:33453)
0809 19:25:05.322628 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:05.322630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:05.322876 (+ 246us) server_negotiation.cc:407] Connection header received
0809 19:25:05.322940 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:05.322941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:05.322951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:05.322988 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:05.324098 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:05.324696 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:05.325604 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:05.325901 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:05.325969 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:05.326671 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:05.326672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:05.326672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:05.326676 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:05.326690 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:05.326694 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:05.326724 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:05.326854 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 19:25:05.326864 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:25:05.336284 10372 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:05.336751 10372 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:05.339851 10372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d0df38750db4794b123c41da4a6b3ca: Generated new TSK 0
I20260809 19:25:05.340010 10372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:05.342046 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10241
W20260809 19:25:05.354214 10218 connection.cc:535] client connection to 127.9.91.126:33453 recv error: Network error: failed to read from TLS socket (remote: 127.9.91.126:33453): Connection reset by peer (error 104)
2026-08-09T19:25:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:05 dist-test-slave-1vmg krb5kdc[10396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:05 dist-test-slave-1vmg krb5kdc[10396](info): set up 2 sockets
Aug 09 19:25:05 dist-test-slave-1vmg krb5kdc[10396](info): commencing operation
krb5kdc: starting...
W20260809 19:25:06.461494 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:06 dist-test-slave-1vmg krb5kdc[10396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:06.569229 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37595
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:06.647504 10411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:06.647681 10411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:06.647703 10411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:06.648761 10411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:06.648794 10411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:06.648808 10411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:06.648818 10411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:06.648833 10411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:06.650198 10411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:37595
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:37595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:06.650382 10411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:06.650545 10411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:06.654884 10424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:06.654891 10420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:06.654891 10427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:06.655160 10411 server_base.cc:1034] running on GCE node
I20260809 19:25:06.655357 10411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:06.655575 10411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:06.656708 10411 hybrid_clock.cc:648] HybridClock initialized: now 1786303506656695 us; error 29 us; skew 500 ppm
Aug 09 19:25:06 dist-test-slave-1vmg krb5kdc[10396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303506, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:06.662497 10411 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:06.662928 10411 webserver.cc:460] Webserver started at http://127.9.91.126:39233/ using document root <none> and password file <none>
I20260809 19:25:06.663139 10411 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:06.663187 10411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:06.663308 10411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:06.664237 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "a5fa8ebe93a54951b2f1a58f9fadc892"
format_stamp: "Formatted at 2026-08-09 19:25:06 on dist-test-slave-1vmg"
server_key: "24b4aa0d709631871da41b66d929fd74"
server_key_iv: "00b41b5de5aa94a9257dafef1ec185e2"
server_key_version: "encryptionkey@0"
I20260809 19:25:06.664549 10411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "a5fa8ebe93a54951b2f1a58f9fadc892"
format_stamp: "Formatted at 2026-08-09 19:25:06 on dist-test-slave-1vmg"
server_key: "24b4aa0d709631871da41b66d929fd74"
server_key_iv: "00b41b5de5aa94a9257dafef1ec185e2"
server_key_version: "encryptionkey@0"
I20260809 19:25:06.665714 10411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:06.666394 10437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:06.666543 10411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:06.666625 10411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "a5fa8ebe93a54951b2f1a58f9fadc892"
format_stamp: "Formatted at 2026-08-09 19:25:06 on dist-test-slave-1vmg"
server_key: "24b4aa0d709631871da41b66d929fd74"
server_key_iv: "00b41b5de5aa94a9257dafef1ec185e2"
server_key_version: "encryptionkey@0"
I20260809 19:25:06.666687 10411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:06.670217 10411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:06.671924 10411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:06.672048 10411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:06.675720 10411 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:37595
I20260809 19:25:06.675758 10523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:37595 every 8 connection(s)
I20260809 19:25:06.676025 10411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:06.676931 10526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:06.679036 10526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892: Bootstrap starting.
I20260809 19:25:06.679582 10526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:06.679838 10526 log.cc:824] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:06.680447 10526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892: No bootstrap required, opened a new log
I20260809 19:25:06.681996 10526 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37595 } }
I20260809 19:25:06.682129 10526 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:06.682157 10526 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5fa8ebe93a54951b2f1a58f9fadc892, State: Initialized, Role: FOLLOWER
I20260809 19:25:06.682267 10526 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37595 } }
I20260809 19:25:06.682350 10526 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:06.682381 10526 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:06.682416 10526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:06.683081 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10411
I20260809 19:25:06.683211 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:06.683350 10526 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37595 } }
I20260809 19:25:06.683414 9581 external_mini_cluster.cc:1428] Setting key 0e9e80275abc1bad378e314cf303d75e
I20260809 19:25:06.683465 10526 leader_election.cc:302] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5fa8ebe93a54951b2f1a58f9fadc892; no voters:
I20260809 19:25:06.683646 10526 leader_election.cc:288] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:06.683691 10533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:06.683874 10533 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [term 1 LEADER]: Becoming Leader. State: Replica: a5fa8ebe93a54951b2f1a58f9fadc892, State: Running, Role: LEADER
I20260809 19:25:06.683890 10526 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:06.684032 10533 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37595 } }
I20260809 19:25:06.684337 10533 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5fa8ebe93a54951b2f1a58f9fadc892. Latest consensus state: current_term: 1 leader_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37595 } } }
I20260809 19:25:06.684401 10533 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:06.684587 10548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:06.685258 10548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:06.685418 10538 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5fa8ebe93a54951b2f1a58f9fadc892" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 37595 } } }
I20260809 19:25:06.685648 10538 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:06.686138 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:06.683917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43687 (local address 127.9.91.126:37595)
0809 19:25:06.684150 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:06.684159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:06.684191 (+ 32us) server_negotiation.cc:407] Connection header received
0809 19:25:06.684298 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:06.684305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:06.684336 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:06.684980 (+ 644us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:06.685658 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:06.685662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:06.685662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:06.685666 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:06.685701 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:06.685716 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:06.685795 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:06.685919 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 19:25:06.685957 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
I20260809 19:25:06.687012 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:06.688534 10548 catalog_manager.cc:1347] Generated new cluster ID: 932dc5119bc642c9a34b13d7438c2e73
I20260809 19:25:06.688581 10548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:06.689649 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:06.688511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46297 (local address 127.9.91.126:37595)
0809 19:25:06.688547 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:06.688549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:06.688809 (+ 260us) server_negotiation.cc:407] Connection header received
0809 19:25:06.688872 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:06.688873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:06.688878 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:06.688930 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:06.689442 (+ 512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:06.689443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:06.689444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:06.689447 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:06.689460 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:06.689464 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:06.689504 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:06.689574 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 19:25:06.689585 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:06.697229 10548 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:06.697381 10548 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:06.706254 10548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5fa8ebe93a54951b2f1a58f9fadc892: Generated new TSK 0
I20260809 19:25:06.706406 10548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:06.707755 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10411
2026-08-09T19:25:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:06 dist-test-slave-1vmg krb5kdc[10580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:06 dist-test-slave-1vmg krb5kdc[10580](info): set up 2 sockets
Aug 09 19:25:06 dist-test-slave-1vmg krb5kdc[10580](info): commencing operation
krb5kdc: starting...
W20260809 19:25:07.843465 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:07 dist-test-slave-1vmg krb5kdc[10580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:07.952796 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46387
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:42045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:46387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:08.033171 10596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:08.033386 10596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:08.033420 10596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:08.034543 10596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:08.034590 10596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:08.034610 10596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:08.034624 10596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:08.034641 10596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:08.036109 10596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:42045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:46387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:46387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:08.036348 10596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:08.036562 10596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:08.041052 10606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:08.041102 10609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:08.041064 10602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:08.041194 10596 server_base.cc:1034] running on GCE node
I20260809 19:25:08.041513 10596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:08.041774 10596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:08.042933 10596 hybrid_clock.cc:648] HybridClock initialized: now 1786303508042918 us; error 32 us; skew 500 ppm
Aug 09 19:25:08 dist-test-slave-1vmg krb5kdc[10580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303508, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:08.045259 10596 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:08.045650 10596 webserver.cc:460] Webserver started at http://127.9.91.126:40381/ using document root <none> and password file <none>
I20260809 19:25:08.045835 10596 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:08.045881 10596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:08.046063 10596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:08.047103 10596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "3767c20cb73a4cfdae38fbc257718538"
format_stamp: "Formatted at 2026-08-09 19:25:08 on dist-test-slave-1vmg"
server_key: "3f4c1fe41f7111093fc497886f836175"
server_key_iv: "419a77cf1f9875d0e5fc19a391f990c5"
server_key_version: "encryptionkey@0"
I20260809 19:25:08.047420 10596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "3767c20cb73a4cfdae38fbc257718538"
format_stamp: "Formatted at 2026-08-09 19:25:08 on dist-test-slave-1vmg"
server_key: "3f4c1fe41f7111093fc497886f836175"
server_key_iv: "419a77cf1f9875d0e5fc19a391f990c5"
server_key_version: "encryptionkey@0"
I20260809 19:25:08.048622 10596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:08.049230 10616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:08.049376 10596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:08.049458 10596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "3767c20cb73a4cfdae38fbc257718538"
format_stamp: "Formatted at 2026-08-09 19:25:08 on dist-test-slave-1vmg"
server_key: "3f4c1fe41f7111093fc497886f836175"
server_key_iv: "419a77cf1f9875d0e5fc19a391f990c5"
server_key_version: "encryptionkey@0"
I20260809 19:25:08.049515 10596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:08.061944 10596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:08.063910 10596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:08.064040 10596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:08.067832 10596 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:46387
I20260809 19:25:08.067867 10689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:46387 every 8 connection(s)
I20260809 19:25:08.068166 10596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:08.068977 10690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:08.071175 10690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538: Bootstrap starting.
I20260809 19:25:08.071688 10690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:08.071928 10690 log.cc:824] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:08.072543 10690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538: No bootstrap required, opened a new log
I20260809 19:25:08.074041 10690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3767c20cb73a4cfdae38fbc257718538" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46387 } }
I20260809 19:25:08.074167 10690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:08.074193 10690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3767c20cb73a4cfdae38fbc257718538, State: Initialized, Role: FOLLOWER
I20260809 19:25:08.074291 10690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3767c20cb73a4cfdae38fbc257718538" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46387 } }
I20260809 19:25:08.074339 10690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:08.074357 10690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:08.074388 10690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:08.075136 10690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3767c20cb73a4cfdae38fbc257718538" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46387 } }
I20260809 19:25:08.075220 10690 leader_election.cc:302] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3767c20cb73a4cfdae38fbc257718538; no voters:
I20260809 19:25:08.075333 10690 leader_election.cc:288] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:08.075374 10693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:08.075500 10693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [term 1 LEADER]: Becoming Leader. State: Replica: 3767c20cb73a4cfdae38fbc257718538, State: Running, Role: LEADER
I20260809 19:25:08.075578 10690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:08.075588 10693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3767c20cb73a4cfdae38fbc257718538" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46387 } }
I20260809 19:25:08.075932 10696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3767c20cb73a4cfdae38fbc257718538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3767c20cb73a4cfdae38fbc257718538" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46387 } } }
I20260809 19:25:08.075948 10701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3767c20cb73a4cfdae38fbc257718538. Latest consensus state: current_term: 1 leader_uuid: "3767c20cb73a4cfdae38fbc257718538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3767c20cb73a4cfdae38fbc257718538" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 46387 } } }
I20260809 19:25:08.076053 10696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:08.076128 10701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:08.076495 10712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:08.076894 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10596
I20260809 19:25:08.077028 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:08.077032 10712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:08.077214 9581 external_mini_cluster.cc:1428] Setting key 156635ce355b3b2315eebda245a94b5f
I20260809 19:25:08.078538 10712 catalog_manager.cc:1347] Generated new cluster ID: 756ef5e4880146b2ab17fa8a057f002b
I20260809 19:25:08.078606 10712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:25:08 dist-test-slave-1vmg krb5kdc[10580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:08.089709 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:08.077767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57779 (local address 127.9.91.126:46387)
0809 19:25:08.077918 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:08.077927 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:08.079133 (+ 1206us) server_negotiation.cc:407] Connection header received
0809 19:25:08.079277 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:08.079284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:08.079335 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:08.079450 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:08.080197 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:08.081574 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:08.082625 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:08.083614 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:08.083706 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:08.086352 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:08.086356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:08.086356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:08.086369 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:08.088188 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:08.088535 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:08.088537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:08.088538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:08.088641 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:08.088811 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:08.088812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:08.088812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:08.089106 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:08.089232 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:08.089338 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 19:25:08.089470 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260809 19:25:08.090209 10712 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:08.090710 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:08.090723 10712 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:08.098512 10712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3767c20cb73a4cfdae38fbc257718538: Generated new TSK 0
I20260809 19:25:08.098661 10712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:08.101683 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:08.095179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42499 (local address 127.9.91.126:46387)
0809 19:25:08.095215 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:08.095216 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:08.095379 (+ 163us) server_negotiation.cc:407] Connection header received
0809 19:25:08.095409 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:08.095410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:08.095431 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:08.095487 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:08.096015 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:08.096762 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:08.097719 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:08.098086 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:08.098153 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:08.099409 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:08.099410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:08.099411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:08.099418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:08.100782 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:08.101062 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:08.101063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:08.101063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:08.101117 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:08.101259 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:08.101260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:08.101260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:08.101371 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:08.101458 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:08.101521 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 19:25:08.101595 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:25:08.103552 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10596
2026-08-09T19:25:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:08 dist-test-slave-1vmg krb5kdc[10740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:08 dist-test-slave-1vmg krb5kdc[10740](info): set up 2 sockets
Aug 09 19:25:08 dist-test-slave-1vmg krb5kdc[10740](info): commencing operation
krb5kdc: starting...
W20260809 19:25:09.236905 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:09 dist-test-slave-1vmg krb5kdc[10740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:09.345189 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:44189
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:44189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:09.423408 10760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:09.423592 10760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:09.423621 10760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:09.424706 10760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:09.424748 10760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:09.424767 10760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:09.424782 10760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:09.424798 10760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:09.426250 10760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:44189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:44189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:09.426447 10760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:09.426602 10760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:09.430888 10770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:09.430943 10768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:09.430987 10767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:09.431203 10760 server_base.cc:1034] running on GCE node
I20260809 19:25:09.431435 10760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:09.431703 10760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:09.432848 10760 hybrid_clock.cc:648] HybridClock initialized: now 1786303509432830 us; error 31 us; skew 500 ppm
Aug 09 19:25:09 dist-test-slave-1vmg krb5kdc[10740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303509, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:09.435359 10760 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:09.435735 10760 webserver.cc:460] Webserver started at http://127.9.91.126:36289/ using document root <none> and password file <none>
I20260809 19:25:09.435918 10760 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:09.435963 10760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:09.436081 10760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:09.437007 10760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "353e509cd8fe49d982a1c191aaa82f1b"
format_stamp: "Formatted at 2026-08-09 19:25:09 on dist-test-slave-1vmg"
server_key: "fa04e3e5f8f9332d6bea8e70b49208a9"
server_key_iv: "36a63a19273d0664bec6fdfd5c4ef66b"
server_key_version: "encryptionkey@0"
I20260809 19:25:09.437318 10760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "353e509cd8fe49d982a1c191aaa82f1b"
format_stamp: "Formatted at 2026-08-09 19:25:09 on dist-test-slave-1vmg"
server_key: "fa04e3e5f8f9332d6bea8e70b49208a9"
server_key_iv: "36a63a19273d0664bec6fdfd5c4ef66b"
server_key_version: "encryptionkey@0"
I20260809 19:25:09.438515 10760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:09.439325 10780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:09.439502 10760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:09.439604 10760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "353e509cd8fe49d982a1c191aaa82f1b"
format_stamp: "Formatted at 2026-08-09 19:25:09 on dist-test-slave-1vmg"
server_key: "fa04e3e5f8f9332d6bea8e70b49208a9"
server_key_iv: "36a63a19273d0664bec6fdfd5c4ef66b"
server_key_version: "encryptionkey@0"
I20260809 19:25:09.439680 10760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:09.470278 10760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:09.472004 10760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:09.472108 10760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:09.475773 10760 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:44189
I20260809 19:25:09.475809 10854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:44189 every 8 connection(s)
I20260809 19:25:09.476095 10760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:09.476869 10855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:09.479003 10855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b: Bootstrap starting.
I20260809 19:25:09.479290 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10760
I20260809 19:25:09.479425 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:09.479607 9581 external_mini_cluster.cc:1428] Setting key d02ec9cfd2d3190741c0a45a9eb82283
I20260809 19:25:09.479696 10855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:09.480042 10855 log.cc:824] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:09.480831 10855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b: No bootstrap required, opened a new log
I20260809 19:25:09.483147 10855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353e509cd8fe49d982a1c191aaa82f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 44189 } }
I20260809 19:25:09.483270 10855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:09.483291 10855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 353e509cd8fe49d982a1c191aaa82f1b, State: Initialized, Role: FOLLOWER
I20260809 19:25:09.483408 10855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353e509cd8fe49d982a1c191aaa82f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 44189 } }
I20260809 19:25:09.483479 10855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:09.483508 10855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:09.483564 10855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:09.484045 10855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353e509cd8fe49d982a1c191aaa82f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 44189 } }
I20260809 19:25:09.484143 10855 leader_election.cc:302] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 353e509cd8fe49d982a1c191aaa82f1b; no voters:
I20260809 19:25:09.484313 10855 leader_election.cc:288] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:09.484334 10861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:09.484457 10861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [term 1 LEADER]: Becoming Leader. State: Replica: 353e509cd8fe49d982a1c191aaa82f1b, State: Running, Role: LEADER
I20260809 19:25:09.484579 10855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:09.484597 10861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353e509cd8fe49d982a1c191aaa82f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 44189 } }
I20260809 19:25:09.484973 10862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "353e509cd8fe49d982a1c191aaa82f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353e509cd8fe49d982a1c191aaa82f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 44189 } } }
I20260809 19:25:09.485060 10862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:09.485230 10863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 353e509cd8fe49d982a1c191aaa82f1b. Latest consensus state: current_term: 1 leader_uuid: "353e509cd8fe49d982a1c191aaa82f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "353e509cd8fe49d982a1c191aaa82f1b" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 44189 } } }
I20260809 19:25:09.485273 10868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:09.485344 10863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:09.485937 10868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:09.486155 10858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:09.480079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38627 (local address 127.9.91.126:44189)
0809 19:25:09.480288 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:09.480296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:09.480331 (+ 35us) server_negotiation.cc:407] Connection header received
0809 19:25:09.480819 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:09.480827 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:09.480887 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:09.481025 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:09.481220 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:09.482581 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:09.483363 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:09.483843 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:09.483941 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:09.485323 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:09.485327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:09.485327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:09.485534 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:09.485572 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:09.485588 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:09.485678 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:09.485866 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 19:25:09.485928 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260809 19:25:09.487147 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:09.488071 10868 catalog_manager.cc:1347] Generated new cluster ID: 859f1810902e46e6b3bd55b6622da9dc
I20260809 19:25:09.488122 10868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:09.493371 10858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:09.488538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37337 (local address 127.9.91.126:44189)
0809 19:25:09.488570 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:09.488572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:09.491072 (+ 2500us) server_negotiation.cc:407] Connection header received
0809 19:25:09.491135 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:09.491136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:09.491146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:09.491192 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:09.491298 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:09.491741 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:09.492378 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:09.492575 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:09.492634 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:09.493161 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:09.493162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:09.493162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:09.493166 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:09.493178 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:09.493181 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:09.493225 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:09.493286 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 19:25:09.493305 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 19:25:09.516305 10868 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:09.516760 10868 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:09.522977 10868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 353e509cd8fe49d982a1c191aaa82f1b: Generated new TSK 0
I20260809 19:25:09.523151 10868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:09.526482 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10760
2026-08-09T19:25:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:09 dist-test-slave-1vmg krb5kdc[10898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:09 dist-test-slave-1vmg krb5kdc[10898](info): set up 2 sockets
Aug 09 19:25:09 dist-test-slave-1vmg krb5kdc[10898](info): commencing operation
krb5kdc: starting...
W20260809 19:25:10.649051 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:10 dist-test-slave-1vmg krb5kdc[10898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:10.758570 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38539
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:39853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:38539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:10.837193 10926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:10.837366 10926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:10.837386 10926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:10.838476 10926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:10.838516 10926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:10.838529 10926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:10.838541 10926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:10.838560 10926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:10.840097 10926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:39853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:38539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:38539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:10.840320 10926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:10.840525 10926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:10.844942 10936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:10.844951 10932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:10.844934 10934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:10.845237 10926 server_base.cc:1034] running on GCE node
I20260809 19:25:10.845470 10926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:10.845727 10926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:10.846864 10926 hybrid_clock.cc:648] HybridClock initialized: now 1786303510846849 us; error 30 us; skew 500 ppm
Aug 09 19:25:10 dist-test-slave-1vmg krb5kdc[10898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303510, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:10.849105 10926 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:10.849480 10926 webserver.cc:460] Webserver started at http://127.9.91.126:34963/ using document root <none> and password file <none>
I20260809 19:25:10.849692 10926 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:10.849740 10926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:10.849859 10926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:10.850782 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "c27710d9c9524b49828562ab6682180c"
format_stamp: "Formatted at 2026-08-09 19:25:10 on dist-test-slave-1vmg"
server_key: "e98ba481027c4ed7999e4f0f21d0bae1"
server_key_iv: "9d5c069767ebe6123c3ca46e75def2b0"
server_key_version: "encryptionkey@0"
I20260809 19:25:10.851120 10926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "c27710d9c9524b49828562ab6682180c"
format_stamp: "Formatted at 2026-08-09 19:25:10 on dist-test-slave-1vmg"
server_key: "e98ba481027c4ed7999e4f0f21d0bae1"
server_key_iv: "9d5c069767ebe6123c3ca46e75def2b0"
server_key_version: "encryptionkey@0"
I20260809 19:25:10.852298 10926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 19:25:10.852941 10944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:10.853082 10926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:10.853164 10926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "c27710d9c9524b49828562ab6682180c"
format_stamp: "Formatted at 2026-08-09 19:25:10 on dist-test-slave-1vmg"
server_key: "e98ba481027c4ed7999e4f0f21d0bae1"
server_key_iv: "9d5c069767ebe6123c3ca46e75def2b0"
server_key_version: "encryptionkey@0"
I20260809 19:25:10.853224 10926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:10.856307 10926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:10.857888 10926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:10.858008 10926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:10.861610 10926 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:38539
I20260809 19:25:10.861653 11016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:38539 every 8 connection(s)
I20260809 19:25:10.861915 10926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:10.862438 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 10926
I20260809 19:25:10.862563 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:10.862738 9581 external_mini_cluster.cc:1428] Setting key c3a18eab285664fdb3b465250bfa90cb
I20260809 19:25:10.862984 11017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:10.864956 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:10.863243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33757 (local address 127.9.91.126:38539)
0809 19:25:10.863412 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:10.863420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:10.863445 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:25:10.863906 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:10.863912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:10.863936 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:10.864038 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:10.864571 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:10.864574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:10.864574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:10.864578 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:10.864610 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:10.864620 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:10.864667 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:10.864744 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 19:25:10.864771 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260809 19:25:10.865788 11017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c: Bootstrap starting.
I20260809 19:25:10.865826 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:10.866436 11017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:10.866765 11017 log.cc:824] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:10.867512 11017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c: No bootstrap required, opened a new log
I20260809 19:25:10.868216 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:10.867329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33913 (local address 127.9.91.126:38539)
0809 19:25:10.867367 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:10.867369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:10.867493 (+ 124us) server_negotiation.cc:407] Connection header received
0809 19:25:10.867558 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:10.867559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:10.867563 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:10.867621 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:10.868045 (+ 424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:10.868046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:10.868047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 19:25:10.868050 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:10.868063 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 19:25:10.868067 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:10.868104 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:10.868159 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 19:25:10.868167 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:25:10.869434 11017 raft_consensus.cc:348] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27710d9c9524b49828562ab6682180c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38539 } }
I20260809 19:25:10.869556 11017 raft_consensus.cc:374] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 19:25:10.869597 11017 raft_consensus.cc:729] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27710d9c9524b49828562ab6682180c, State: Initialized, Role: FOLLOWER
I20260809 19:25:10.869711 11017 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27710d9c9524b49828562ab6682180c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38539 } }
I20260809 19:25:10.869797 11017 raft_consensus.cc:388] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 19:25:10.869829 11017 raft_consensus.cc:482] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 19:25:10.869875 11017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:10.870340 11017 raft_consensus.cc:504] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27710d9c9524b49828562ab6682180c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38539 } }
I20260809 19:25:10.870433 11017 leader_election.cc:302] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c27710d9c9524b49828562ab6682180c; no voters:
I20260809 19:25:10.870582 11017 leader_election.cc:288] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 19:25:10.870631 11032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:10.870774 11032 raft_consensus.cc:686] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [term 1 LEADER]: Becoming Leader. State: Replica: c27710d9c9524b49828562ab6682180c, State: Running, Role: LEADER
I20260809 19:25:10.870810 11017 sys_catalog.cc:564] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:10.870901 11032 consensus_queue.cc:237] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27710d9c9524b49828562ab6682180c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38539 } }
I20260809 19:25:10.871277 11033 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c27710d9c9524b49828562ab6682180c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27710d9c9524b49828562ab6682180c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38539 } } }
I20260809 19:25:10.871387 11033 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:10.871279 11034 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c27710d9c9524b49828562ab6682180c. Latest consensus state: current_term: 1 leader_uuid: "c27710d9c9524b49828562ab6682180c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c27710d9c9524b49828562ab6682180c" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 38539 } } }
I20260809 19:25:10.871472 11034 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:10.871649 11047 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:10.872123 11047 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:10.873531 11047 catalog_manager.cc:1347] Generated new cluster ID: c36a0c23e10f44ef8768d3cc029d0032
I20260809 19:25:10.873590 11047 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:10.880721 11047 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:10.880856 11047 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:10.886487 11047 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c27710d9c9524b49828562ab6682180c: Generated new TSK 0
I20260809 19:25:10.886638 11047 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:10.895228 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 10926
2026-08-09T19:25:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1360 ms)
[----------] 18 tests from AuthTokenIssuingTest (25097 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:10 dist-test-slave-1vmg krb5kdc[11065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:10 dist-test-slave-1vmg krb5kdc[11065](info): set up 2 sockets
Aug 09 19:25:10 dist-test-slave-1vmg krb5kdc[11065](info): commencing operation
krb5kdc: starting...
W20260809 19:25:12.017007 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:12.123878 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36701
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36701,127.9.91.125:41081,127.9.91.124:38405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:12.202816 11085 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:25:12.203059 11085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:12.203083 11085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:12.203104 11085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:12.204200 11085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:12.204238 11085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:12.204254 11085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:12.204273 11085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:12.204290 11085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:12.205746 11085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36701,127.9.91.125:41081,127.9.91.124:38405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:36701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:12.205940 11085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:12.206120 11085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:12.210798 11096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:12.210873 11085 server_base.cc:1034] running on GCE node
W20260809 19:25:12.210810 11091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:12.210817 11093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:12.211278 11085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:12.211544 11085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:12.212677 11085 hybrid_clock.cc:648] HybridClock initialized: now 1786303512212664 us; error 30 us; skew 500 ppm
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:12.218596 11085 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:12.219004 11085 webserver.cc:460] Webserver started at http://127.9.91.126:44373/ using document root <none> and password file <none>
I20260809 19:25:12.219218 11085 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:12.219266 11085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:12.219386 11085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:12.220342 11085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "58219cf7fa314cc0b7341bcfd5a6075f"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "fcab0865309dd6fde14c1d1ccb4c865b"
server_key_iv: "34c9dbe7f2c041313f47a0ae8b06c34a"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.220656 11085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "58219cf7fa314cc0b7341bcfd5a6075f"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "fcab0865309dd6fde14c1d1ccb4c865b"
server_key_iv: "34c9dbe7f2c041313f47a0ae8b06c34a"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.221834 11085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:12.222462 11107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:12.222599 11085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 19:25:12.222682 11085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "58219cf7fa314cc0b7341bcfd5a6075f"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "fcab0865309dd6fde14c1d1ccb4c865b"
server_key_iv: "34c9dbe7f2c041313f47a0ae8b06c34a"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.222739 11085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:12.236838 11085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:12.238413 11085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:12.238545 11085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:12.242326 11085 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:36701
I20260809 19:25:12.242363 11186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:36701 every 8 connection(s)
I20260809 19:25:12.242654 11085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:12.243616 11187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:12.245057 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:12.247853 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11085
I20260809 19:25:12.247959 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:12.248162 9581 external_mini_cluster.cc:1428] Setting key d681224f1ab7fcd7cb663736e166ac71
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:12.253713 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.245679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.9.91.126:36701)
0809 19:25:12.245815 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.245823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.245930 (+ 107us) server_negotiation.cc:407] Connection header received
0809 19:25:12.246416 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.246417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.246434 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.246507 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.247026 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.247798 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.248714 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.249009 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.249105 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.250939 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.250946 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.250946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.250953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.252310 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.252638 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.252641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.252642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.252713 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.252897 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.252898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.252899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.253088 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.253155 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.253331 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.253514 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260809 19:25:12.253716 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.245639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36701 (local address 127.0.0.1:35650)
0809 19:25:12.245803 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.245839 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.246288 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.246528 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.246530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.246752 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.246899 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.246903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.247883 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.247884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.248637 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.248640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.249105 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.249106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.249168 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.249627 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.249635 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.250845 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.252385 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.252390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.252392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.252573 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.252768 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.252769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.252769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.252846 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.253149 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.253154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.253213 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.253287 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.253456 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260809 19:25:12.254451 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:12.254729 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.254627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44646)
0809 19:25:12.254664 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.254705 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 19:25:12.254845 11109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
I20260809 19:25:12.255227 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.255142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44650)
0809 19:25:12.255182 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.255205 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 19:25:12.255311 11187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:41081: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.91.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab.
Entry for principal kudu/127.9.91.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab.
I20260809 19:25:12.271051 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:41081
--webserver_interface=127.9.91.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab
--principal=kudu/127.9.91.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36701,127.9.91.125:41081,127.9.91.124:38405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:25:12.304489 11187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } attempt: 1
I20260809 19:25:12.304939 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.304820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44666)
0809 19:25:12.304877 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.304913 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:25:12.305292 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.305211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44672)
0809 19:25:12.305251 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.305275 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 19:25:12.305382 11187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:41081: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
W20260809 19:25:12.351811 11197 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:25:12.352001 11197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:12.352020 11197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:12.352041 11197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:12.353094 11197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:12.353132 11197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:12.353147 11197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:12.353159 11197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:12.353175 11197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:12.354580 11197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36701,127.9.91.125:41081,127.9.91.124:38405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab
--principal=kudu/127.9.91.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:41081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.91.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:12.354756 11197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:12.354964 11197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:12.359283 11208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:12.359289 11205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:12.359354 11206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:12.359644 11197 server_base.cc:1034] running on GCE node
I20260809 19:25:12.359877 11197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:12.360163 11197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:12.361303 11197 hybrid_clock.cc:648] HybridClock initialized: now 1786303512361288 us; error 29 us; skew 500 ppm
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:12.363720 11197 init.cc:377] Logged in from keytab as kudu/127.9.91.125@KRBTEST.COM (short username kudu)
I20260809 19:25:12.364130 11197 webserver.cc:460] Webserver started at http://127.9.91.125:38305/ using document root <none> and password file <none>
I20260809 19:25:12.364343 11197 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:12.364395 11197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:12.364527 11197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:12.365494 11197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data/instance:
uuid: "aa87b22cc9444d6aa92a4189e854ae6e"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "7ccb063292b5f1fa0ec5261e0f8d1e0a"
server_key_iv: "146eb26745a9bb7cd3b8d9109e0ed41c"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.365834 11197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/wal/instance:
uuid: "aa87b22cc9444d6aa92a4189e854ae6e"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "7ccb063292b5f1fa0ec5261e0f8d1e0a"
server_key_iv: "146eb26745a9bb7cd3b8d9109e0ed41c"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.367139 11197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 19:25:12.367807 11220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:12.367957 11197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:12.368050 11197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/wal
uuid: "aa87b22cc9444d6aa92a4189e854ae6e"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "7ccb063292b5f1fa0ec5261e0f8d1e0a"
server_key_iv: "146eb26745a9bb7cd3b8d9109e0ed41c"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.368115 11197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:12.373538 11187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } attempt: 2
I20260809 19:25:12.373853 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.373756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44688)
0809 19:25:12.373802 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.373828 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:25:12.374143 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.374065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44702)
0809 19:25:12.374102 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.374125 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:25:12.374215 11187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:41081: Network error: Client connection negotiation failed: client connection to 127.9.91.125:41081: connect: Connection refused (error 111)
I20260809 19:25:12.383065 11197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:12.385103 11197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:12.385241 11197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:12.389140 11197 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.125:41081
I20260809 19:25:12.389168 11301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.125:41081 every 8 connection(s)
I20260809 19:25:12.389468 11197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
I20260809 19:25:12.390338 11302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:12.391703 11302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:12.395068 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11197
I20260809 19:25:12.395176 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-1/wal/instance
I20260809 19:25:12.395366 9581 external_mini_cluster.cc:1428] Setting key 56e12c18b89fdbd024ef0c3425a73420
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:12.399928 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.392260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.9.91.126:36701)
0809 19:25:12.392307 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.392309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.392509 (+ 200us) server_negotiation.cc:407] Connection header received
0809 19:25:12.392965 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.392966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.392974 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.393029 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.393552 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.394005 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.394897 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.395215 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.395279 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.397381 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.397383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.397383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.397391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.398593 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.398957 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.398959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.398959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.399019 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.399259 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.399261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.399261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.399422 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.399509 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.399727 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.399837 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:12.399935 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.392235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36701 (local address 127.0.0.1:35656)
0809 19:25:12.392399 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.392438 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.392876 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.393078 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.393081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.393298 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.393476 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.393481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.394075 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.394076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.394827 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.394830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.395321 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.395322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.395420 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.395836 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.395844 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.397292 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.398666 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.398673 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.398675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.398876 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.399073 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.399074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.399074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.399173 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.399494 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.399498 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.399605 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.399681 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.399799 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260809 19:25:12.400516 11302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.91.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.124@KRBTEST.COM" created.
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:25:12.409996 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.400738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44706)
0809 19:25:12.400783 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.400793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.400832 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.401044 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.401046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.401185 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.401249 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.401251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.402023 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.402024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.402751 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.402754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.403257 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.403259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.403334 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.403649 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.403654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.407296 (+ 3642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.409101 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.409102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.409103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.409258 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.409476 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.409477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.409477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.409526 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.409802 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.409803 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.409834 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.409877 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.409937 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:25:12.410132 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.400789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.9.91.125:41081)
0809 19:25:12.400912 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.400916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.400933 (+ 17us) server_negotiation.cc:407] Connection header received
0809 19:25:12.400953 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.400955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.400964 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.401029 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.401318 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.401934 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.402835 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.403181 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.403234 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.407387 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.407396 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.407396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.407402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.409017 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.409339 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.409342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.409342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.409411 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.409566 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.409566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.409567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.409739 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.409829 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.409915 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.410025 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260809 19:25:12.410688 11302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:12.410984 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.410868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55884)
0809 19:25:12.410908 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.410966 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:25:12.411120 11224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
I20260809 19:25:12.411533 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.411446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55888)
0809 19:25:12.411487 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.411517 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 19:25:12.411639 11302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:38405: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab.
Entry for principal kudu/127.9.91.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab.
I20260809 19:25:12.420711 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:38405
--webserver_interface=127.9.91.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab
--principal=kudu/127.9.91.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:36701,127.9.91.125:41081,127.9.91.124:38405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:25:12.460868 11302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } attempt: 1
I20260809 19:25:12.461292 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.461174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55900)
0809 19:25:12.461230 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.461266 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 19:25:12.461606 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.461537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55916)
0809 19:25:12.461569 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.461592 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:25:12.461691 11302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:38405: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
I20260809 19:25:12.465337 11187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } attempt: 3
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:25:12.472417 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.465663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.9.91.125:41081)
0809 19:25:12.465707 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.465709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.465722 (+ 13us) server_negotiation.cc:407] Connection header received
0809 19:25:12.465804 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.465805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.465812 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.465865 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.466242 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.466714 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.467733 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.467940 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.467988 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.469943 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.469944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.469944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.469950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.471320 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.471700 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.471701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.471701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.471750 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.471959 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.471960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.471960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.472074 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.472125 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.472304 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.472366 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 19:25:12.472465 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.465628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44718)
0809 19:25:12.465687 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.465697 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.465746 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.465886 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.465887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.466084 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.466161 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.466163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.466780 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.466781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.467644 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.467648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.468012 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.468013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.468094 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.468489 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.468494 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.469834 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.471399 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.471401 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.471402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.471629 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.471807 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.471807 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.471808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.471900 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.472134 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.472135 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.472183 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.472263 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.472362 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 19:25:12.472898 11187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:12.473220 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.473104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55932)
0809 19:25:12.473152 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.473190 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:25:12.473518 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.473433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55948)
0809 19:25:12.473473 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.473499 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 19:25:12.473605 11187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:38405: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
W20260809 19:25:12.503072 11313 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:25:12.503245 11313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:12.503261 11313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:12.503279 11313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:12.504344 11313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:12.504388 11313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:12.504411 11313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:12.504437 11313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:12.504454 11313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:12.504737 11187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } attempt: 1
I20260809 19:25:12.505081 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.504969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55956)
0809 19:25:12.505018 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.505055 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:25:12.505383 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.505313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55962)
0809 19:25:12.505345 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.505366 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:25:12.505462 11187 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:38405: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
I20260809 19:25:12.506199 11313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:36701,127.9.91.125:41081,127.9.91.124:38405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab
--principal=kudu/127.9.91.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:38405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.91.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:12.506428 11313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:12.506632 11313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:12.510933 11324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:12.510982 11319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:12.510982 11318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:12.511286 11313 server_base.cc:1034] running on GCE node
I20260809 19:25:12.511511 11313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:12.511777 11313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:12.512923 11313 hybrid_clock.cc:648] HybridClock initialized: now 1786303512512906 us; error 26 us; skew 500 ppm
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:12.515180 11313 init.cc:377] Logged in from keytab as kudu/127.9.91.124@KRBTEST.COM (short username kudu)
I20260809 19:25:12.515592 11313 webserver.cc:460] Webserver started at http://127.9.91.124:39381/ using document root <none> and password file <none>
I20260809 19:25:12.515806 11313 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:12.515853 11313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:12.515977 11313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:12.516929 11313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data/instance:
uuid: "acfcfaf350e0441292dd52dfdbbd1923"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "3944a2a79c25ea1dc20510989c5df7cb"
server_key_iv: "1f75d78e9b7dbf4599a9e3e2c038534a"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.517256 11313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/wal/instance:
uuid: "acfcfaf350e0441292dd52dfdbbd1923"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "3944a2a79c25ea1dc20510989c5df7cb"
server_key_iv: "1f75d78e9b7dbf4599a9e3e2c038534a"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.519069 11313 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 19:25:12.519797 11335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:12.519965 11313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:12.520069 11313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/wal
uuid: "acfcfaf350e0441292dd52dfdbbd1923"
format_stamp: "Formatted at 2026-08-09 19:25:12 on dist-test-slave-1vmg"
server_key: "3944a2a79c25ea1dc20510989c5df7cb"
server_key_iv: "1f75d78e9b7dbf4599a9e3e2c038534a"
server_key_version: "encryptionkey@0"
I20260809 19:25:12.520141 11313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:12.529838 11302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } attempt: 2
I20260809 19:25:12.530186 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.530073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55972)
0809 19:25:12.530124 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.530160 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:25:12.530467 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.530400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55974)
0809 19:25:12.530430 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.530451 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 19:25:12.530544 11302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:38405: Network error: Client connection negotiation failed: client connection to 127.9.91.124:38405: connect: Connection refused (error 111)
I20260809 19:25:12.541232 11313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:12.542784 11313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:12.542896 11313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:12.546825 11313 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.124:38405
I20260809 19:25:12.546854 11408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.124:38405 every 8 connection(s)
I20260809 19:25:12.547195 11313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
I20260809 19:25:12.548076 11409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:12.549423 11409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:12.555039 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11313
I20260809 19:25:12.555153 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786303248613589-9581-0/minicluster-data/master-2/wal/instance
I20260809 19:25:12.555318 9581 external_mini_cluster.cc:1428] Setting key 136e888db60fc037e82f3ab2b677dde1
I20260809 19:25:12.557451 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.550022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.9.91.126:36701)
0809 19:25:12.550109 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.550111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.550262 (+ 151us) server_negotiation.cc:407] Connection header received
0809 19:25:12.550739 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.550740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.550752 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.550803 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.551421 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.551888 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.552808 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.552998 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.553053 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.554838 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.554840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.554840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.554848 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.556240 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.556582 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.556584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.556584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.556648 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.556892 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.556893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.556893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.556998 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.557052 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.557331 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.557392 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 19:25:12.557799 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.549959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:36701 (local address 127.0.0.1:35670)
0809 19:25:12.550144 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.550184 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.550643 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.550864 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.550867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.551155 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.551335 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.551339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.551968 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.551969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.552726 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.552728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.553087 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.553088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.553185 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.553550 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.553558 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.554754 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.556318 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.556322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.556323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.556522 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.556709 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.556709 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.556709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.556832 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.557058 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.557063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.557208 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.557309 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.557607 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":149,"thread_start_us":76,"threads_started":1}
I20260809 19:25:12.558334 11409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:25:12.565425 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.558543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:41081 (local address 127.0.0.1:44724)
0809 19:25:12.558602 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.558610 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.558644 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.558763 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.558763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.558861 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.558903 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.558904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.559698 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.559699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.560173 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.560175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.560664 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.560665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.560731 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.561035 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.561040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.562341 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.564349 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.564350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.564350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.564583 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.564785 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.564785 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.564786 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.564843 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.565147 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.565148 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.565206 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.565262 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.565343 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 19:25:12.565562 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.558601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44724 (local address 127.9.91.125:41081)
0809 19:25:12.558662 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.558665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.558672 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:25:12.558695 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.558696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.558705 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.558760 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.558991 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.559629 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.560247 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.560547 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.560612 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.562440 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.562441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.562441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.562447 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.564274 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.564646 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.564648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.564648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.564717 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.564951 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.564952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.564952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.565091 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.565154 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.565327 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.565423 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 19:25:12.565804 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.555661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.9.91.124:38405)
0809 19:25:12.555802 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.555806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.555943 (+ 137us) server_negotiation.cc:407] Connection header received
0809 19:25:12.555994 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.555996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.556010 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.556098 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.556753 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.557688 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.558580 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.558950 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.559009 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.561422 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.561431 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.561431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.561438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.563924 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.564330 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.564333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.564333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.564400 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.564613 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.564614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.564614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.564787 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.564928 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.565331 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.565409 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260809 19:25:12.565847 11409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:25:12.573998 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.566133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:55990)
0809 19:25:12.566188 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.566196 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.566235 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.566366 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.566367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.566499 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.566526 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.566528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.567349 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.567350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.567954 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.567956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.568287 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.568288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.568317 (+ 29us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.569884 (+ 1567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.569888 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.571155 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.572929 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.572930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.572930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.573125 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.573321 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.573321 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.573322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.573387 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.573686 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.573687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.573730 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.573801 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.573890 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:25:12.574100 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.566170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55990 (local address 127.9.91.124:38405)
0809 19:25:12.566201 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.566203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.566228 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:25:12.566291 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.566292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.566299 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.566335 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.566587 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.567274 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.568018 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.568221 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.568257 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.571219 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.571220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.571220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.571226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.572830 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.573211 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.573212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.573213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.573262 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.573464 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.573465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.573465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.573585 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.573657 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.573844 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.573952 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":17}
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:12.575695 11409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: Bootstrap starting.
I20260809 19:25:12.576648 11409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:12.576936 11409 log.cc:824] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:12.577672 11409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: No bootstrap required, opened a new log
I20260809 19:25:12.578079 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.570768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.9.91.126:36701)
0809 19:25:12.570809 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.570811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.570889 (+ 78us) server_negotiation.cc:407] Connection header received
0809 19:25:12.571014 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.571015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.571025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.571081 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.571599 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.572203 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.573136 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.573404 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.573459 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.575578 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.575579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.575580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.575586 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.577081 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.577399 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.577400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.577400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.577453 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.577623 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.577624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.577624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.577743 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.577898 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.577934 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.577997 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 19:25:12.580113 11409 raft_consensus.cc:348] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.580317 11409 raft_consensus.cc:729] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acfcfaf350e0441292dd52dfdbbd1923, State: Initialized, Role: FOLLOWER
I20260809 19:25:12.580427 11409 consensus_queue.cc:260] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.580610 11187 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } attempt: 2
I20260809 19:25:12.580770 11417 sys_catalog.cc:455] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.580858 11417 sys_catalog.cc:458] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:12.580818 11409 sys_catalog.cc:564] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 19:25:12.583489 11435 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:25:12.583945 11313 master_runner.cc:186] Error getting master registration for 127.9.91.126:36701: OK, instance_id { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" instance_seqno: 1786303512241288 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260809 19:25:12.585958 11435 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:12.586035 11313 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:25:12.589066 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.578700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.9.91.125:41081)
0809 19:25:12.578741 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.578742 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.578748 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:25:12.578764 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.578765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.578772 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.578815 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.579429 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.579990 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.581443 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.581728 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.581793 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.584279 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.584280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.584280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.584286 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.587971 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.588378 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.588379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.588379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.588439 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.588630 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.588631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.588631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.588740 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.588888 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.588923 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.588987 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 19:25:12.590296 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.581009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56000 (local address 127.9.91.124:38405)
0809 19:25:12.581046 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.581047 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.581056 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:25:12.581075 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.581076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.581085 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.581144 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.581420 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.582067 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.583037 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.583309 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.583369 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.587618 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.587620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.587620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.587626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.589127 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.589485 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.589486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.589487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.589534 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.589807 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.589808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.589808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.589950 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.590007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.590143 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.590215 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:25:12.590301 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.580801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:56000)
0809 19:25:12.580841 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.580849 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.580879 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.581161 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.581162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.581304 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.581340 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.581341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.582138 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.582139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.582943 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.582946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.583565 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.583566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.583626 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.586259 (+ 2633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.586263 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.587519 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.589209 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.589210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.589210 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.589407 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.589693 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.589694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.589695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.589758 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.590015 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.590016 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.590057 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.590117 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.590215 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 19:25:12.591434 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:12.592049 11187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f: Bootstrap starting.
I20260809 19:25:12.592748 11187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:12.593040 11187 log.cc:824] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:12.592387 11146 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "58219cf7fa314cc0b7341bcfd5a6075f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35684
I20260809 19:25:12.593806 11187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f: No bootstrap required, opened a new log
I20260809 19:25:12.595060 11187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.595180 11187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58219cf7fa314cc0b7341bcfd5a6075f, State: Initialized, Role: FOLLOWER
I20260809 19:25:12.595256 11187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.596414 11187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:12.596447 11146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 19:25:12.596488 11146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:12.597188 11146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.597409 11146 leader_election.cc:288] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [CANDIDATE]: Term 1 election: Requested vote from peers aa87b22cc9444d6aa92a4189e854ae6e (127.9.91.125:41081), acfcfaf350e0441292dd52dfdbbd1923 (127.9.91.124:38405)
I20260809 19:25:12.596390 11438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.599119 11438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:12.599148 11373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "acfcfaf350e0441292dd52dfdbbd1923"
I20260809 19:25:12.599277 11373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:12.600124 11373 raft_consensus.cc:2454] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58219cf7fa314cc0b7341bcfd5a6075f in term 1.
I20260809 19:25:12.600776 11108 leader_election.cc:302] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58219cf7fa314cc0b7341bcfd5a6075f, acfcfaf350e0441292dd52dfdbbd1923; no voters:
I20260809 19:25:12.600899 11438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:12.601039 11438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [term 1 LEADER]: Becoming Leader. State: Replica: 58219cf7fa314cc0b7341bcfd5a6075f, State: Running, Role: LEADER
I20260809 19:25:12.601120 11438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.601464 11438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58219cf7fa314cc0b7341bcfd5a6075f. Latest consensus state: current_term: 1 leader_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.601549 11438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:12.599790 11085 master_runner.cc:186] Error getting master registration for 127.9.91.125:41081: OK, instance_id { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" instance_seqno: 1786303512388047 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:25:12.601869 11085 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:25:12.606585 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.598665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56008 (local address 127.9.91.124:38405)
0809 19:25:12.598703 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.598705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.599257 (+ 552us) server_negotiation.cc:407] Connection header received
0809 19:25:12.599281 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.599281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.599290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.599343 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.600062 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.600727 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.601693 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.601947 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.601991 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.603467 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.603469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.603469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.603476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.604974 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.605534 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.605535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.605535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.605591 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.605927 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.605928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.605928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.606051 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.606117 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.606438 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.606502 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 19:25:12.607386 11463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:12.607909 11463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:12.608421 11310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.598624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44744 (local address 127.9.91.125:41081)
0809 19:25:12.598658 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.598659 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.600234 (+ 1575us) server_negotiation.cc:407] Connection header received
0809 19:25:12.600257 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.600258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.600267 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.600329 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.600995 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.601622 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.602655 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.603418 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.603483 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.605088 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.605089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.605089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.605095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.606591 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.607397 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.607398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.607398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.607609 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.607812 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.607813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.607813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.607929 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.608049 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.608266 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.608335 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:25:12.611797 11373 raft_consensus.cc:1261] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [term 1 FOLLOWER]: Refusing update from remote peer 58219cf7fa314cc0b7341bcfd5a6075f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:12.612493 11461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:12.614408 11417 sys_catalog.cc:455] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58219cf7fa314cc0b7341bcfd5a6075f. Latest consensus state: current_term: 1 leader_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.614456 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.607136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.9.91.126:36701)
0809 19:25:12.607174 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.607176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.607397 (+ 221us) server_negotiation.cc:407] Connection header received
0809 19:25:12.607535 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.607536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.607559 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.607588 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.608386 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.609034 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.610035 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.610345 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.610397 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.611643 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.611644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.611644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.611651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.613272 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.613692 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.613693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.613693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.613763 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.613956 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.613957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.613957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.614071 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.614158 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.614312 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.614378 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:12.614493 11417 sys_catalog.cc:458] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:12.621678 11302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } attempt: 3
I20260809 19:25:12.622891 11438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.623013 11438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:12.623284 11417 sys_catalog.cc:455] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.623369 11417 sys_catalog.cc:458] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:12.623678 11463 catalog_manager.cc:1347] Generated new cluster ID: afddf180234c42e38e42a1f82328ce3c
I20260809 19:25:12.623714 11463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 19:25:12 dist-test-slave-1vmg krb5kdc[11065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303512, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:25:12.626984 11465 mvcc.cc:205] Tried to move back new op lower bound from 7316699187655401472 to 7316699187614781440. Current Snapshot: MvccSnapshot[applied={T|T < 7316699187655401472 or (T in {7316699187655401472})}]
I20260809 19:25:12.628793 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.621896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:38405 (local address 127.0.0.1:56016)
0809 19:25:12.621938 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:12.621947 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:12.621977 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:12.622103 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:12.622104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:12.622239 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:12.622269 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.622270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.623098 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.623099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.623848 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:12.623850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.624232 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.624232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:12.624287 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.624621 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:12.624626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:12.626094 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:12.627790 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.627791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.627792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.628000 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.628194 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:12.628194 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:12.628195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:12.628270 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:12.628520 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:12.628521 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:12.628559 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 19:25:12.628617 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 19:25:12.628702 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:25:12.628803 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:12.621927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.9.91.124:38405)
0809 19:25:12.621975 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.621977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.621984 (+ 7us) server_negotiation.cc:407] Connection header received
0809 19:25:12.622024 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.622025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.622033 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.622082 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.622336 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.623013 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.623921 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:12.624172 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:12.624228 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:12.626175 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:12.626176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:12.626177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:12.626183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:12.627714 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.628080 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.628081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.628081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.628147 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:12.628314 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:12.628314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:12.628315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:12.628442 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:12.628511 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:12.628670 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 19:25:12.628730 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:25:12.628850 11466 mvcc.cc:205] Tried to move back new op lower bound from 7316699187655401472 to 7316699187614781440. Current Snapshot: MvccSnapshot[applied={T|T < 7316699187655401472}]
I20260809 19:25:12.630254 11302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: Bootstrap starting.
I20260809 19:25:12.630561 11463 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:12.630995 11302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:12.631250 11463 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:12.631271 11302 log.cc:824] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:12.631981 11302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: No bootstrap required, opened a new log
I20260809 19:25:12.633893 11302 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.634007 11302 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa87b22cc9444d6aa92a4189e854ae6e, State: Initialized, Role: FOLLOWER
I20260809 19:25:12.634068 11302 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } }
I20260809 19:25:12.634341 11302 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [sys.catalog]: configured and running, proceeding with master startup.
W20260809 19:25:12.635739 11233 tablet.cc:2369] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 19:25:12.637305 11489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:12.637403 11489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:12.637564 11473 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.637694 11473 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:12.639284 11263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:12.640157 11263 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [term 1 FOLLOWER]: Refusing update from remote peer 58219cf7fa314cc0b7341bcfd5a6075f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:12.640326 11265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "aa87b22cc9444d6aa92a4189e854ae6e"
I20260809 19:25:12.640419 11461 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:12.641057 11265 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58219cf7fa314cc0b7341bcfd5a6075f in term 1.
I20260809 19:25:12.643229 11473 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.643327 11473 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:12.643591 11497 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58219cf7fa314cc0b7341bcfd5a6075f. Latest consensus state: current_term: 1 leader_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.643661 11497 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:12.643730 11463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f: Generated new TSK 0
I20260809 19:25:12.643872 11463 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:12.659168 11438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58219cf7fa314cc0b7341bcfd5a6075f" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 36701 } } peers { permanent_uuid: "aa87b22cc9444d6aa92a4189e854ae6e" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 41081 } } peers { permanent_uuid: "acfcfaf350e0441292dd52dfdbbd1923" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 38405 } } }
I20260809 19:25:12.659255 11438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58219cf7fa314cc0b7341bcfd5a6075f [sys.catalog]: This master's current role is: LEADER
W20260809 19:25:12.675595 11191 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.673944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.9.91.126:36701)
0809 19:25:12.673977 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.673979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.674514 (+ 535us) server_negotiation.cc:407] Connection header received
0809 19:25:12.675066 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.675067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.675077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.675105 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.675541 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35692: BlockingRecv error: recv got EOF from 127.0.0.1:35692 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:25:12.676147 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.673767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.9.91.125:41081)
0809 19:25:12.673802 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.673803 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.674645 (+ 842us) server_negotiation.cc:407] Connection header received
0809 19:25:12.675734 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.675735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.675746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.675775 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.676084 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44760: BlockingRecv error: recv got EOF from 127.0.0.1:44760 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:25:12.676522 11413 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.673491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56030 (local address 127.9.91.124:38405)
0809 19:25:12.673528 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.673530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.674427 (+ 897us) server_negotiation.cc:407] Connection header received
0809 19:25:12.674690 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.674691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.674702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.674729 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.676471 (+ 1742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56030: BlockingRecv error: recv got EOF from 127.0.0.1:56030 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 19:25:12.678766 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.678052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.9.91.125:41081)
0809 19:25:12.678084 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.678086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.678295 (+ 209us) server_negotiation.cc:407] Connection header received
0809 19:25:12.678401 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.678402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.678417 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.678444 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.678725 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44776: BlockingRecv error: recv got EOF from 127.0.0.1:44776 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:25:12.687922 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.687243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.9.91.125:41081)
0809 19:25:12.687274 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.687276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.687482 (+ 206us) server_negotiation.cc:407] Connection header received
0809 19:25:12.687566 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.687567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.687576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.687601 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.687884 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44790: BlockingRecv error: recv got EOF from 127.0.0.1:44790 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:25:12.691654 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.690990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44806 (local address 127.9.91.125:41081)
0809 19:25:12.691019 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.691021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.691230 (+ 209us) server_negotiation.cc:407] Connection header received
0809 19:25:12.691313 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.691314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.691322 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.691348 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.691618 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44806: BlockingRecv error: recv got EOF from 127.0.0.1:44806 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:25:12.697367 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.696656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.9.91.125:41081)
0809 19:25:12.696687 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.696688 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.696917 (+ 229us) server_negotiation.cc:407] Connection header received
0809 19:25:12.697002 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.697003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.697011 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.697037 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.697328 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44808: BlockingRecv error: recv got EOF from 127.0.0.1:44808 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:25:12.707083 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.706364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44812 (local address 127.9.91.125:41081)
0809 19:25:12.706394 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.706396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.706599 (+ 203us) server_negotiation.cc:407] Connection header received
0809 19:25:12.706685 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.706686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.706695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.706720 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.707043 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44812: BlockingRecv error: recv got EOF from 127.0.0.1:44812 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 19:25:12.724982 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.724186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44820 (local address 127.9.91.125:41081)
0809 19:25:12.724218 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.724219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.724448 (+ 229us) server_negotiation.cc:407] Connection header received
0809 19:25:12.724542 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.724543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.724553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.724581 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.724944 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44820: BlockingRecv error: recv got EOF from 127.0.0.1:44820 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:25:12.758945 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.758090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44822 (local address 127.9.91.125:41081)
0809 19:25:12.758126 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.758127 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.758399 (+ 272us) server_negotiation.cc:407] Connection header received
0809 19:25:12.758501 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.758501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.758511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.758540 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.758876 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44822: BlockingRecv error: recv got EOF from 127.0.0.1:44822 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:25:12.824791 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.824121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.9.91.125:41081)
0809 19:25:12.824175 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.824178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.824284 (+ 106us) server_negotiation.cc:407] Connection header received
0809 19:25:12.824348 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.824349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.824361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.824406 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.824729 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44824: BlockingRecv error: recv got EOF from 127.0.0.1:44824 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 19:25:12.954456 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:12.953787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.9.91.125:41081)
0809 19:25:12.953878 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:12.953880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:12.953946 (+ 66us) server_negotiation.cc:407] Connection header received
0809 19:25:12.954017 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:12.954018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:12.954029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:12.954077 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:12.954400 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44840: BlockingRecv error: recv got EOF from 127.0.0.1:44840 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 19:25:13.212031 11310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:13.211519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.9.91.125:41081)
0809 19:25:13.211575 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:13.211576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:13.211585 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:25:13.211647 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:13.211647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:13.211656 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:13.211710 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:13.211981 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44848: BlockingRecv error: recv got EOF from 127.0.0.1:44848 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 19:25:13.586381 11435 catalog_manager.cc:1259] Loaded cluster ID: afddf180234c42e38e42a1f82328ce3c
I20260809 19:25:13.586448 11435 catalog_manager.cc:1552] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: loading cluster ID for follower catalog manager: success
I20260809 19:25:13.587262 11435 catalog_manager.cc:1574] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: acquiring CA information for follower catalog manager: success
I20260809 19:25:13.587548 11435 catalog_manager.cc:1602] T 00000000000000000000000000000000 P acfcfaf350e0441292dd52dfdbbd1923: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:25:13.637825 11489 catalog_manager.cc:1259] Loaded cluster ID: afddf180234c42e38e42a1f82328ce3c
I20260809 19:25:13.637883 11489 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: loading cluster ID for follower catalog manager: success
I20260809 19:25:13.638680 11489 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: acquiring CA information for follower catalog manager: success
I20260809 19:25:13.638985 11489 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aa87b22cc9444d6aa92a4189e854ae6e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:25:13.727475 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:13.725127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.9.91.125:41081)
0809 19:25:13.725240 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:13.725242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:13.725253 (+ 11us) server_negotiation.cc:407] Connection header received
0809 19:25:13.725327 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:13.725328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:13.725375 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:13.725447 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:13.725569 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.726079 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.726828 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.727042 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.727088 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:13.727204 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:13.727311 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:13.727363 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:13.727378 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 19:25:13.727394 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260809 19:25:13.731426 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:13.729303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.9.91.124:38405)
0809 19:25:13.729434 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:13.729436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:13.729461 (+ 25us) server_negotiation.cc:407] Connection header received
0809 19:25:13.729497 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:13.729498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:13.729533 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:13.729592 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:13.729669 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.730149 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.730780 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.731026 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.731078 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:13.731183 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:13.731274 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:13.731336 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:13.731349 (+ 13us) server_negotiation.cc:299] Negotiation successful
0809 19:25:13.731366 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260809 19:25:13.735627 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:13.732943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44862 (local address 127.9.91.125:41081)
0809 19:25:13.733026 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:13.733028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:13.733038 (+ 10us) server_negotiation.cc:407] Connection header received
0809 19:25:13.733133 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:13.733134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:13.733148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:13.733200 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:13.733393 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.734102 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.734958 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.735157 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.735210 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:13.735389 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:13.735423 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:13.735478 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:13.735535 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 19:25:13.735556 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 19:25:13.736187 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:13.732987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56054 (local address 127.9.91.124:38405)
0809 19:25:13.733041 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:13.733043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:13.733133 (+ 90us) server_negotiation.cc:407] Connection header received
0809 19:25:13.733184 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:13.733185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:13.733194 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:13.733236 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:13.733353 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.734067 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.735273 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.735553 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.735607 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:13.735936 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:13.735967 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:13.736014 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:13.736096 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 19:25:13.736121 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 19:25:13.736225 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:13.732853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.9.91.126:36701)
0809 19:25:13.732969 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:13.732972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:13.733253 (+ 281us) server_negotiation.cc:407] Connection header received
0809 19:25:13.733302 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:13.733303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:13.733358 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:13.733428 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:13.733579 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.734100 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.735347 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:13.735673 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:13.735735 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:13.735930 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:13.736024 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:13.736084 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:13.736134 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 19:25:13.736157 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260809 19:25:13.737663 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11085
I20260809 19:25:13.753949 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11197
I20260809 19:25:13.764518 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11313
2026-08-09T19:25:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2869 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 19:25:13 dist-test-slave-1vmg krb5kdc[11620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 19:25:13 dist-test-slave-1vmg krb5kdc[11620](info): set up 2 sockets
Aug 09 19:25:13 dist-test-slave-1vmg krb5kdc[11620](info): commencing operation
krb5kdc: starting...
W20260809 19:25:14.893786 9581 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/test-user.keytab.
Aug 09 19:25:14 dist-test-slave-1vmg krb5kdc[11620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T19:25:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T19:25:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.91.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
Entry for principal kudu/127.9.91.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab.
I20260809 19:25:15.003119 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35749
--webserver_interface=127.9.91.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:35749,127.9.91.125:36341,127.9.91.124:41691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 19:25:15.084291 11639 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:25:15.084493 11639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:15.084513 11639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:15.084533 11639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:15.085711 11639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:15.085760 11639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:15.085776 11639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:15.085789 11639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:15.085801 11639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:15.087318 11639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:35749,127.9.91.125:36341,127.9.91.124:41691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.126.keytab
--principal=kudu/127.9.91.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.126:35749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.91.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:15.087551 11639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:15.087771 11639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:15.092352 11646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:15.092391 11651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:15.092422 11639 server_base.cc:1034] running on GCE node
W20260809 19:25:15.092406 11645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:15.092810 11639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:15.093106 11639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:15.094237 11639 hybrid_clock.cc:648] HybridClock initialized: now 1786303515094223 us; error 34 us; skew 500 ppm
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:15.096625 11639 init.cc:377] Logged in from keytab as kudu/127.9.91.126@KRBTEST.COM (short username kudu)
I20260809 19:25:15.096974 11639 webserver.cc:460] Webserver started at http://127.9.91.126:43593/ using document root <none> and password file <none>
I20260809 19:25:15.097143 11639 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:15.097179 11639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:15.097266 11639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:15.098179 11639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data/instance:
uuid: "257690616a404c0f9789b07f67b6a593"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "17ef85fb9007645b66cd0d705bc664bb"
server_key_iv: "c6ac3250d4b2d69934e0d486d86f4eec"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.098454 11639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/wal/instance:
uuid: "257690616a404c0f9789b07f67b6a593"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "17ef85fb9007645b66cd0d705bc664bb"
server_key_iv: "c6ac3250d4b2d69934e0d486d86f4eec"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.099651 11639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:15.100334 11660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:15.100502 11639 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:15.100585 11639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/wal
uuid: "257690616a404c0f9789b07f67b6a593"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "17ef85fb9007645b66cd0d705bc664bb"
server_key_iv: "c6ac3250d4b2d69934e0d486d86f4eec"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.100636 11639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:15.110011 11639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:15.111917 11639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:15.112020 11639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:15.115684 11639 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.126:35749
I20260809 19:25:15.115737 11722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.126:35749 every 8 connection(s)
I20260809 19:25:15.116014 11639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/data/info.pb
I20260809 19:25:15.116957 11723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:15.118041 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11639
I20260809 19:25:15.118131 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-0/wal/instance
I20260809 19:25:15.118301 9581 external_mini_cluster.cc:1428] Setting key 3dc5afd1ba2d4e714ce7275a71ec4e91
I20260809 19:25:15.118557 11723 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.91.125@KRBTEST.COM; defaulting to no policy
I20260809 19:25:15.128885 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.119182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35749 (local address 127.0.0.1:45028)
0809 19:25:15.119346 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.119389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.119950 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.120253 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.120256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.120464 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.120609 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.120613 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.121819 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.121821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.122844 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.122847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.123417 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.123418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.123500 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.123981 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.123991 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.125636 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.127412 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.127419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.127421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.127608 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.127878 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.127879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.127879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.127958 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.128278 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.128283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.128344 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.128421 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.128581 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260809 19:25:15.128887 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.119217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.9.91.126:35749)
0809 19:25:15.119356 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.119365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.119462 (+ 97us) server_negotiation.cc:407] Connection header received
0809 19:25:15.120118 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.120120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.120145 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.120250 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.120700 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.121719 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.122967 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.123299 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.123391 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.125725 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.125734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.125734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.125741 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.127321 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.127718 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.127721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.127721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.127797 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.128015 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.128016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.128016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.128217 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.128287 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.128464 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.128644 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.125@KRBTEST.COM" created.
I20260809 19:25:15.129688 11723 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:15.129989 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.129890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59016)
0809 19:25:15.129930 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.129970 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 19:25:15.130101 11664 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
I20260809 19:25:15.130461 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.130383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59024)
0809 19:25:15.130418 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.130440 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 19:25:15.130550 11723 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:36341: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab.
Entry for principal kudu/127.9.91.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab.
I20260809 19:25:15.141098 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:36341
--webserver_interface=127.9.91.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab
--principal=kudu/127.9.91.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:35749,127.9.91.125:36341,127.9.91.124:41691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:25:15.179757 11723 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } attempt: 1
I20260809 19:25:15.180143 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.180029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59040)
0809 19:25:15.180082 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.180121 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:25:15.180460 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.180369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59050)
0809 19:25:15.180416 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.180440 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 19:25:15.180545 11723 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:36341: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
W20260809 19:25:15.219498 11734 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:25:15.219681 11734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:15.219712 11734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:15.219738 11734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:15.220832 11734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:15.220872 11734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:15.220891 11734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:15.220906 11734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:15.220922 11734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:15.222344 11734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:35749,127.9.91.125:36341,127.9.91.124:41691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.125.keytab
--principal=kudu/127.9.91.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.125:36341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.91.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:15.222564 11734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:15.222766 11734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:15.227190 11744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:15.227243 11748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:15.227217 11745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:15.227375 11734 server_base.cc:1034] running on GCE node
I20260809 19:25:15.227586 11734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:15.227844 11734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:15.228967 11734 hybrid_clock.cc:648] HybridClock initialized: now 1786303515228960 us; error 26 us; skew 500 ppm
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:15.231185 11734 init.cc:377] Logged in from keytab as kudu/127.9.91.125@KRBTEST.COM (short username kudu)
I20260809 19:25:15.231575 11734 webserver.cc:460] Webserver started at http://127.9.91.125:32925/ using document root <none> and password file <none>
I20260809 19:25:15.231779 11734 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:15.231825 11734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:15.231943 11734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:15.232868 11734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data/instance:
uuid: "80dbcffec8864a99b6777511a7870118"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "a41fa409bc59b33406359c45777e3b53"
server_key_iv: "770bac85557697016972fddc0a9ff3d7"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.233165 11734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/wal/instance:
uuid: "80dbcffec8864a99b6777511a7870118"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "a41fa409bc59b33406359c45777e3b53"
server_key_iv: "770bac85557697016972fddc0a9ff3d7"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.234308 11734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 19:25:15.234954 11759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:15.235103 11734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:15.235183 11734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/wal
uuid: "80dbcffec8864a99b6777511a7870118"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "a41fa409bc59b33406359c45777e3b53"
server_key_iv: "770bac85557697016972fddc0a9ff3d7"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.235245 11734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:15.248667 11723 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } attempt: 2
I20260809 19:25:15.248988 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.248886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59054)
0809 19:25:15.248934 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.248961 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:25:15.249297 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.249215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59070)
0809 19:25:15.249255 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.249279 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 19:25:15.249382 11723 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.125:36341: Network error: Client connection negotiation failed: client connection to 127.9.91.125:36341: connect: Connection refused (error 111)
I20260809 19:25:15.252609 11734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:15.254069 11734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:15.254191 11734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:15.257794 11734 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.125:36341
I20260809 19:25:15.257833 11837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.125:36341 every 8 connection(s)
I20260809 19:25:15.258114 11734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/data/info.pb
I20260809 19:25:15.258998 11838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:15.260321 11838 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:15.264979 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11734
I20260809 19:25:15.265066 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-1/wal/instance
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:15.265249 9581 external_mini_cluster.cc:1428] Setting key 8e358e239673991e2c1fb66f5d541179
I20260809 19:25:15.268019 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.260862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.9.91.126:35749)
0809 19:25:15.260914 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.260916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.261087 (+ 171us) server_negotiation.cc:407] Connection header received
0809 19:25:15.261496 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.261496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.261505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.261547 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.262044 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.262503 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.263420 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.263608 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.263655 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.265501 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.265502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.265503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.265510 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.266793 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.267183 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.267184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.267185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.267242 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.267449 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.267450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.267450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.267561 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.267628 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.267852 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.267937 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 19:25:15.268167 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.260843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35749 (local address 127.0.0.1:45034)
0809 19:25:15.260987 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.261018 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.261426 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.261603 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.261606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.261814 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.261987 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.261990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.262557 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.262558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.263359 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.263362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.263679 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.263679 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.263769 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.264167 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.264175 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.265420 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.266873 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.266878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.266880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.267112 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.267297 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.267298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.267298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.267391 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.267618 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.267625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.267739 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.267822 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.267960 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260809 19:25:15.268693 11838 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.91.124@KRBTEST.COM; defaulting to no policy
I20260809 19:25:15.275233 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.268889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59074)
0809 19:25:15.268932 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.268943 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.268983 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.269206 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.269207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.269358 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.269412 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.269414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.270213 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.270214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.270976 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.270979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.271380 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.271381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.271450 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.271754 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.271759 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.272865 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.274336 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.274337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.274338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.274482 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.274659 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.274659 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.274660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.274711 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.274948 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.274948 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.274984 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.275044 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.275144 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:25:15.275375 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.268930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.9.91.125:36341)
0809 19:25:15.269074 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.269079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.269096 (+ 17us) server_negotiation.cc:407] Connection header received
0809 19:25:15.269116 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.269118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.269128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.269184 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.269479 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.270135 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.271042 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.271316 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.271364 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.272935 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.272943 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.272944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.272950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.274264 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.274546 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.274548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.274549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.274604 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.274759 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.274760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.274761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.274876 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.274959 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.275078 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.275195 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260809 19:25:15.275843 11838 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.91.124@KRBTEST.COM" created.
I20260809 19:25:15.276125 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.276015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44388)
0809 19:25:15.276052 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.276104 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 19:25:15.276262 11765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
I20260809 19:25:15.276602 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.276539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44402)
0809 19:25:15.276563 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.276584 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 19:25:15.276692 11838 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:41691: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.91.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab.
Entry for principal kudu/127.9.91.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab.
I20260809 19:25:15.287468 9581 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskt8E36c/build/release/bin/kudu
/tmp/dist-test-taskt8E36c/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:41691
--webserver_interface=127.9.91.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab
--principal=kudu/127.9.91.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.91.84:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.91.126:35749,127.9.91.125:36341,127.9.91.124:41691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 19:25:15.325896 11838 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } attempt: 1
I20260809 19:25:15.326373 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.326210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44414)
0809 19:25:15.326281 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.326339 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 19:25:15.326856 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.326738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44422)
0809 19:25:15.326802 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.326831 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 19:25:15.326969 11838 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:41691: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
I20260809 19:25:15.340512 11723 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } attempt: 3
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:25:15.347316 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.340843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.9.91.125:36341)
0809 19:25:15.340886 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.340888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.340900 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:25:15.340970 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.340971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.340983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.341039 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.341313 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.342093 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.342745 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.343100 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.343174 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.345069 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.345070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.345071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.345077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.346300 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.346671 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.346671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.346672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.346720 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.346891 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.346891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.346892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.347007 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.347057 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.347206 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.347262 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:25:15.347371 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.340804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59082)
0809 19:25:15.340856 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.340867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.340916 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.341043 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.341044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.341184 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.341257 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.341259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.342167 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.342167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.342680 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.342682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.343167 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.343168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.343232 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.343617 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.343621 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.344995 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.346378 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.346379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.346380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.346605 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.346771 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.346772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.346772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.346841 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.347060 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.347061 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.347111 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.347185 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.347277 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 19:25:15.347808 11723 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:15.348109 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.348015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44436)
0809 19:25:15.348055 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.348086 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:25:15.348418 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.348346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44444)
0809 19:25:15.348381 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.348403 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 19:25:15.348500 11723 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:41691: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
W20260809 19:25:15.369764 11848 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 19:25:15.369933 11848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 19:25:15.369962 11848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 19:25:15.369995 11848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 19:25:15.371119 11848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 19:25:15.371156 11848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 19:25:15.371168 11848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 19:25:15.371192 11848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 19:25:15.371215 11848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 19:25:15.372628 11848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.91.84:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.91.126:35749,127.9.91.125:36341,127.9.91.124:41691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/krb5kdc/kudu_127.9.91.124.keytab
--principal=kudu/127.9.91.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.91.124:41691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.91.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 18:26:05 UTC on ccycloud.quasar-yhjrge.root.comops.site
build id 4576684
I20260809 19:25:15.372835 11848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 19:25:15.373028 11848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 19:25:15.377211 11855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:15.377326 11858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 19:25:15.377293 11854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 19:25:15.377578 11848 server_base.cc:1034] running on GCE node
I20260809 19:25:15.377789 11848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 19:25:15.378039 11848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 19:25:15.379168 11848 hybrid_clock.cc:648] HybridClock initialized: now 1786303515379149 us; error 31 us; skew 500 ppm
I20260809 19:25:15.379624 11723 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } attempt: 1
I20260809 19:25:15.379923 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.379832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44446)
0809 19:25:15.379873 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.379900 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 19:25:15.380241 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.380149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44460)
0809 19:25:15.380191 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.380216 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 19:25:15.380326 11723 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:41691: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 19:25:15.381546 11848 init.cc:377] Logged in from keytab as kudu/127.9.91.124@KRBTEST.COM (short username kudu)
I20260809 19:25:15.381916 11848 webserver.cc:460] Webserver started at http://127.9.91.124:36489/ using document root <none> and password file <none>
I20260809 19:25:15.382117 11848 fs_manager.cc:359] Metadata directory not provided
I20260809 19:25:15.382164 11848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 19:25:15.382288 11848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 19:25:15.383314 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data/instance:
uuid: "5d91fbb1a9184fe3a05e016b6524b7ff"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "1fa2b186ade3bbafc991c49d82933613"
server_key_iv: "85a8086a8ae99b7eee33fa1b9c1a379f"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.383613 11848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/wal/instance:
uuid: "5d91fbb1a9184fe3a05e016b6524b7ff"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "1fa2b186ade3bbafc991c49d82933613"
server_key_iv: "85a8086a8ae99b7eee33fa1b9c1a379f"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.384760 11848 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 19:25:15.385361 11870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 19:25:15.385504 11848 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 19:25:15.385588 11848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data,/tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/wal
uuid: "5d91fbb1a9184fe3a05e016b6524b7ff"
format_stamp: "Formatted at 2026-08-09 19:25:15 on dist-test-slave-1vmg"
server_key: "1fa2b186ade3bbafc991c49d82933613"
server_key_iv: "85a8086a8ae99b7eee33fa1b9c1a379f"
server_key_version: "encryptionkey@0"
I20260809 19:25:15.385648 11848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 19:25:15.395081 11838 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } attempt: 2
I20260809 19:25:15.395377 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.395286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44474)
0809 19:25:15.395331 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.395355 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 19:25:15.395675 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.395610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44490)
0809 19:25:15.395641 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.395662 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 19:25:15.395748 11838 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.91.124:41691: Network error: Client connection negotiation failed: client connection to 127.9.91.124:41691: connect: Connection refused (error 111)
I20260809 19:25:15.402259 11848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 19:25:15.404212 11848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 19:25:15.404314 11848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 19:25:15.407902 11848 rpc_server.cc:307] RPC server started. Bound to: 127.9.91.124:41691
I20260809 19:25:15.407932 11941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.91.124:41691 every 8 connection(s)
I20260809 19:25:15.408226 11848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/data/info.pb
I20260809 19:25:15.409109 11943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 19:25:15.410485 11943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:15.411335 9581 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskt8E36c/build/release/bin/kudu as pid 11848
I20260809 19:25:15.411439 9581 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskt8E36c/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786303248613589-9581-0/minicluster-data/master-2/wal/instance
I20260809 19:25:15.411619 9581 external_mini_cluster.cc:1428] Setting key 35889bac87c99185e3bbeeb7a8b91c39
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:25:15.419603 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.411079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.9.91.126:35749)
0809 19:25:15.411127 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.411129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.411322 (+ 193us) server_negotiation.cc:407] Connection header received
0809 19:25:15.411840 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.411841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.411852 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.411903 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.412607 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.413181 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.414421 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.414609 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.414672 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.416764 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.416765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.416765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.416772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.418280 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.418699 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.418700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.418701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.418758 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.419017 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.419018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.419018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.419120 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.419187 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.419445 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.419520 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 19:25:15.419847 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.411064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.126:35749 (local address 127.0.0.1:45042)
0809 19:25:15.411219 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.411252 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.411754 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.411973 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.411977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.412288 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.412505 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.412512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.413258 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.413259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.414342 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.414345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.414711 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.414712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.414803 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.415273 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.415283 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.416678 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.418365 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.418371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.418373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.418632 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.418816 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.418816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.418817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.418962 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.419183 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.419188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.419315 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.419419 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.419527 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":188,"thread_start_us":73,"threads_started":1}
I20260809 19:25:15.420013 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.411955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.9.91.124:41691)
0809 19:25:15.412076 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.412080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.412118 (+ 38us) server_negotiation.cc:407] Connection header received
0809 19:25:15.412177 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.412179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.412196 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.412263 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.412909 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.413795 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.414644 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.414999 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.415065 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.417245 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.417253 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.417254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.417260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.418800 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.419151 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.419154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.419155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.419210 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.419389 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.419390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.419390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.419535 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.419808 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.419848 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.419912 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260809 19:25:15.420570 11943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.126@KRBTEST.COM
I20260809 19:25:15.426612 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.420794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.125:36341 (local address 127.0.0.1:59086)
0809 19:25:15.420836 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.420845 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.420874 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.420976 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.420977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.421120 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.421153 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.421154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.421919 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.421919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.422592 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.422595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.422994 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.422994 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.423042 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.423364 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.423369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.424464 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.425790 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.425791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.425791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.425940 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.426104 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.426104 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.426105 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.426158 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.426381 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.426382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.426423 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.426474 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.426538 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 19:25:15.426667 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.420848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.9.91.125:36341)
0809 19:25:15.420888 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.420889 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.420897 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:15.420919 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.420920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.420929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.420976 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.421222 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.421849 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.422662 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.422869 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.423000 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.424539 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.424541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.424541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.424547 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.425721 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.425998 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.425999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.426000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.426053 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.426207 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.426208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.426209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.426309 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.426369 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.426511 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.426578 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:25:15.426998 11943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } has no permanent_uuid. Determining permanent_uuid...
I20260809 19:25:15.427609 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.420665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.9.91.126:35749)
0809 19:25:15.420702 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.420703 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.420877 (+ 174us) server_negotiation.cc:407] Connection header received
0809 19:25:15.420899 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.420900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.420909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.420963 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.421429 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.422034 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.422859 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.423129 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.423187 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.425085 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.425086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.425086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.425097 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.426285 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.426612 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.426613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.426613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.427001 (+ 388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.427177 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.427178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.427178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.427282 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.427434 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.427470 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.427531 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.124@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.91.125@KRBTEST.COM
I20260809 19:25:15.433516 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.427197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44502)
0809 19:25:15.427273 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.427280 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.427318 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.427409 (+ 91us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.427410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.427527 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.427550 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.427551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.428271 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.428272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.428935 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.428937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.429356 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.429357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.429396 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.429693 (+ 297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.429697 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.430886 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.432256 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.432257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.432257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.432445 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.432627 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.432627 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.432627 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.432675 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.432875 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.432876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.433086 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.433155 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.433403 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 19:25:15.433581 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.427226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.9.91.124:41691)
0809 19:25:15.427254 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.427256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.427306 (+ 50us) server_negotiation.cc:407] Connection header received
0809 19:25:15.427357 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.427357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.427366 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.427408 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.427614 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.428198 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.428997 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.429296 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.429359 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.430985 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.430986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.430986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.430992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.432175 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.432509 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.432510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.432510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.432563 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.432729 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.432730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.432730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.432821 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.432875 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.433190 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.433470 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":214,"server-negotiator.queue_time_us":13}
I20260809 19:25:15.434648 11943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: Bootstrap starting.
I20260809 19:25:15.434844 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.428155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59092 (local address 127.9.91.125:36341)
0809 19:25:15.428191 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.428192 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.428200 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:15.428218 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.428219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.428227 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.428271 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.428787 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.429346 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.430181 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.430475 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.430544 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.432734 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.432735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.432735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.432741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.433883 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.434158 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.434159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.434160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.434210 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.434366 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.434367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.434368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.434464 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.434590 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.434628 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.434688 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 19:25:15.435539 11943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:15.435822 11943 log.cc:824] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:15.436127 9581 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 19:25:15.436527 11943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: No bootstrap required, opened a new log
I20260809 19:25:15.437209 11692 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "257690616a404c0f9789b07f67b6a593"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45052
I20260809 19:25:15.438038 11943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
I20260809 19:25:15.438148 11943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d91fbb1a9184fe3a05e016b6524b7ff, State: Initialized, Role: FOLLOWER
I20260809 19:25:15.438241 11943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
I20260809 19:25:15.438511 11943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:15.438500 11954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.438604 11954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:25:15.440109 11971 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:15.440161 11971 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:15.440358 11848 master_runner.cc:186] Error getting master registration for 127.9.91.126:35749: OK, instance_id { permanent_uuid: "257690616a404c0f9789b07f67b6a593" instance_seqno: 1786303515114642 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:25:15.442458 11848 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:25:15.455423 11723 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } attempt: 2
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.126@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:25:15.462519 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.455703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.9.91.124:41691)
0809 19:25:15.455753 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.455754 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.455763 (+ 9us) server_negotiation.cc:407] Connection header received
0809 19:25:15.455815 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.455816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.455824 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.455876 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.456142 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.456810 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.457727 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.458012 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.458065 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.459988 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.459989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.459989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.459995 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.461395 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.461687 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.461688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.461688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.461740 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.462006 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.462006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.462006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.462153 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.462211 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.462365 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.462434 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 19:25:15.462548 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.455647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44514)
0809 19:25:15.455708 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.455720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.455765 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.455889 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.455889 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.456042 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.456079 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.456080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.456877 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.456878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.457653 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.457655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.458070 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.458070 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.458126 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.458511 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.458515 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.459899 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.461458 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.461459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.461459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.461625 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.461835 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.461837 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.461837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.461929 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.462215 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.462216 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.462264 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.462341 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.462443 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 19:25:15.463971 11723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593: Bootstrap starting.
I20260809 19:25:15.464656 11723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:15.464941 11723 log.cc:824] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:15.465670 11723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593: No bootstrap required, opened a new log
I20260809 19:25:15.466638 11723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
I20260809 19:25:15.466740 11723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257690616a404c0f9789b07f67b6a593, State: Initialized, Role: FOLLOWER
I20260809 19:25:15.466813 11723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
I20260809 19:25:15.467077 11977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.467144 11723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:15.467208 11977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:15.467233 11692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 19:25:15.467381 11692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:15.468154 11692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
W20260809 19:25:15.469041 11991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 19:25:15.469002 11904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "257690616a404c0f9789b07f67b6a593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff"
I20260809 19:25:15.469111 11904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [term 0 FOLLOWER]: Advancing to term 1
W20260809 19:25:15.469156 11991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:15.469997 11904 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257690616a404c0f9789b07f67b6a593 in term 1.
I20260809 19:25:15.470299 11665 leader_election.cc:302] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 257690616a404c0f9789b07f67b6a593, 5d91fbb1a9184fe3a05e016b6524b7ff; no voters:
I20260809 19:25:15.470427 11977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 19:25:15.470582 11977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [term 1 LEADER]: Becoming Leader. State: Replica: 257690616a404c0f9789b07f67b6a593, State: Running, Role: LEADER
I20260809 19:25:15.470669 11977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
I20260809 19:25:15.471097 11977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 257690616a404c0f9789b07f67b6a593. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.471201 11977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:15.469475 11639 master_runner.cc:186] Error getting master registration for 127.9.91.125:36341: OK, instance_id { permanent_uuid: "80dbcffec8864a99b6777511a7870118" instance_seqno: 1786303515256814 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 19:25:15.471563 11639 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 19:25:15.468401 11692 leader_election.cc:288] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [CANDIDATE]: Term 1 election: Requested vote from peers 80dbcffec8864a99b6777511a7870118 (127.9.91.125:36341), 5d91fbb1a9184fe3a05e016b6524b7ff (127.9.91.124:41691)
I20260809 19:25:15.475485 11999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 19:25:15.475696 11999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 19:25:15.477082 11904 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [term 1 FOLLOWER]: Refusing update from remote peer 257690616a404c0f9789b07f67b6a593: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:15.477401 11977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 19:25:15.478585 11954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 257690616a404c0f9789b07f67b6a593. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.478690 11954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:15.486611 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.475745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44528 (local address 127.9.91.124:41691)
0809 19:25:15.475798 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.475800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.478873 (+ 3073us) server_negotiation.cc:407] Connection header received
0809 19:25:15.479320 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.479321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.479333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.479381 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.480011 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.480660 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.481601 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.481869 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.481909 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.483425 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.483426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.483426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.483433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.485315 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.485779 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.485781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.485781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.485840 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.486050 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.486051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.486051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.486179 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.486242 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.486453 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.486526 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 19:25:15.486795 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.472538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.9.91.126:35749)
0809 19:25:15.472596 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.472598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.472856 (+ 258us) server_negotiation.cc:407] Connection header received
0809 19:25:15.472957 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.472957 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.472967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.473043 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.474217 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.474882 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.476743 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.477030 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.477073 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.480468 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.480469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.480470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.480477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.482262 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.483852 (+ 1590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.483853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.483853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.483913 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.485723 (+ 1810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.485736 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.485736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.485869 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.485965 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.486627 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.486705 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:15.486932 11838 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } attempt: 3
I20260809 19:25:15.487293 11977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.487373 11977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: This master's current role is: LEADER
I20260809 19:25:15.487560 12011 mvcc.cc:205] Tried to move back new op lower bound from 7316699199392169984 to 7316699199368572928. Current Snapshot: MvccSnapshot[applied={T|T < 7316699199392169984}]
I20260809 19:25:15.487813 11954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.487882 11954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:15.488250 12014 mvcc.cc:205] Tried to move back new op lower bound from 7316699199392169984 to 7316699199368572928. Current Snapshot: MvccSnapshot[applied={T|T < 7316699199392169984}]
Aug 09 19:25:15 dist-test-slave-1vmg krb5kdc[11620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786303515, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.91.125@KRBTEST.COM for kudu/127.9.91.124@KRBTEST.COM
I20260809 19:25:15.495095 11840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.487200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.91.124:41691 (local address 127.0.0.1:44544)
0809 19:25:15.487281 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 19:25:15.487290 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 19:25:15.487326 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 19:25:15.487599 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 19:25:15.487600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 19:25:15.487761 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 19:25:15.487804 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.487805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.488738 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.488740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.489490 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 19:25:15.489502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.490024 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.490025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 19:25:15.490098 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.490542 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 19:25:15.490546 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 19:25:15.492060 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 19:25:15.493860 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.493861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.493861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.494084 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.494340 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 19:25:15.494341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 19:25:15.494341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 19:25:15.494414 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 19:25:15.494740 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 19:25:15.494741 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 19:25:15.494797 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 19:25:15.494863 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 19:25:15.494996 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 19:25:15.495465 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.487413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.9.91.124:41691)
0809 19:25:15.487456 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.487458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.487464 (+ 6us) server_negotiation.cc:407] Connection header received
0809 19:25:15.487482 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.487483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.487504 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.487557 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.487890 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.488584 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.489613 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.489899 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.489959 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.492172 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.492173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.492173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.492180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.493743 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.494183 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.494184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.494185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.494255 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.494503 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.494503 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.494504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.494636 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.494698 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.495310 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.495384 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:15.496625 11842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:15.472735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59096 (local address 127.9.91.125:36341)
0809 19:25:15.472773 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.472774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.473017 (+ 243us) server_negotiation.cc:407] Connection header received
0809 19:25:15.473123 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.473124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.473134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.473163 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.473833 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.474534 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.475560 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:15.475830 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:15.475873 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:15.483564 (+ 7691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 19:25:15.483565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 19:25:15.483566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 19:25:15.483573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 19:25:15.485227 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.490366 (+ 5139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.490367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.490367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.490431 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 19:25:15.495892 (+ 5461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 19:25:15.495893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 19:25:15.495894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 19:25:15.496031 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 19:25:15.496126 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:15.496435 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 19:25:15.496528 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 19:25:15.496841 11838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: Bootstrap starting.
I20260809 19:25:15.497514 11838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: Neither blocks nor log segments found. Creating new log.
I20260809 19:25:15.497612 11999 catalog_manager.cc:1347] Generated new cluster ID: 75ba29908e6149a391fa39ebaa212c0d
I20260809 19:25:15.497663 11999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 19:25:15.497794 11838 log.cc:824] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: Log is configured to *not* fsync() on all Append() calls
I20260809 19:25:15.499068 11838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: No bootstrap required, opened a new log
I20260809 19:25:15.501029 11838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
I20260809 19:25:15.501168 11838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80dbcffec8864a99b6777511a7870118, State: Initialized, Role: FOLLOWER
I20260809 19:25:15.501245 11838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } }
I20260809 19:25:15.501528 11838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 19:25:15.501609 11798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "257690616a404c0f9789b07f67b6a593" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "80dbcffec8864a99b6777511a7870118"
I20260809 19:25:15.501734 11798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [term 0 FOLLOWER]: Advancing to term 1
I20260809 19:25:15.502528 11798 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 257690616a404c0f9789b07f67b6a593 in term 1.
I20260809 19:25:15.502998 11797 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [term 1 FOLLOWER]: Refusing update from remote peer 257690616a404c0f9789b07f67b6a593: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 19:25:15.503640 11994 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 19:25:15.504675 11767 tablet.cc:2369] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 19:25:15.505076 12040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.505172 12040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 19:25:15.505964 12038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 19:25:15.506019 12038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 19:25:15.507658 12024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 257690616a404c0f9789b07f67b6a593. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.507743 12024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:15.507793 12043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.507851 12043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 19:25:15.512112 12041 mvcc.cc:205] Tried to move back new op lower bound from 7316699199392169984 to 7316699199368572928. Current Snapshot: MvccSnapshot[applied={T|T < 7316699199392169984}]
I20260809 19:25:15.514979 11999 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 19:25:15.515653 11999 catalog_manager.cc:1504] Loading token signing keys...
I20260809 19:25:15.519403 11999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593: Generated new TSK 0
I20260809 19:25:15.519556 11999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 19:25:15.521461 11994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "257690616a404c0f9789b07f67b6a593" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "257690616a404c0f9789b07f67b6a593" member_type: VOTER last_known_addr { host: "127.9.91.126" port: 35749 } } peers { permanent_uuid: "80dbcffec8864a99b6777511a7870118" member_type: VOTER last_known_addr { host: "127.9.91.125" port: 36341 } } peers { permanent_uuid: "5d91fbb1a9184fe3a05e016b6524b7ff" member_type: VOTER last_known_addr { host: "127.9.91.124" port: 41691 } } }
I20260809 19:25:15.521544 11994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257690616a404c0f9789b07f67b6a593 [sys.catalog]: This master's current role is: LEADER
W20260809 19:25:15.543263 11949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.541612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44546 (local address 127.9.91.124:41691)
0809 19:25:15.541650 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.541652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.542584 (+ 932us) server_negotiation.cc:407] Connection header received
0809 19:25:15.542691 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.542692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.542702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.542731 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.543206 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44546: BlockingRecv error: recv got EOF from 127.0.0.1:44546 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:25:15.544245 11727 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.541973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.9.91.126:35749)
0809 19:25:15.542010 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.542012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.543576 (+ 1564us) server_negotiation.cc:407] Connection header received
0809 19:25:15.543731 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.543732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.543743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.543772 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.544190 (+ 418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45056: BlockingRecv error: recv got EOF from 127.0.0.1:45056 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 19:25:15.544952 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.542259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.9.91.125:36341)
0809 19:25:15.542299 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.542300 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.543858 (+ 1558us) server_negotiation.cc:407] Connection header received
0809 19:25:15.544550 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.544551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.544561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.544590 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.544901 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59110: BlockingRecv error: recv got EOF from 127.0.0.1:59110 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 19:25:15.547298 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.546522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.9.91.125:36341)
0809 19:25:15.546567 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.546569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.546773 (+ 204us) server_negotiation.cc:407] Connection header received
0809 19:25:15.546872 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.546873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.546888 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.546948 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.547260 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59124: BlockingRecv error: recv got EOF from 127.0.0.1:59124 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260809 19:25:15.550186 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.549439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.9.91.125:36341)
0809 19:25:15.549470 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.549472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.549724 (+ 252us) server_negotiation.cc:407] Connection header received
0809 19:25:15.549814 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.549815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.549825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.549852 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.550150 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59126: BlockingRecv error: recv got EOF from 127.0.0.1:59126 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:25:15.553957 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.553221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59128 (local address 127.9.91.125:36341)
0809 19:25:15.553251 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.553253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.553477 (+ 224us) server_negotiation.cc:407] Connection header received
0809 19:25:15.553584 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.553585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.553595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.553622 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.553919 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59128: BlockingRecv error: recv got EOF from 127.0.0.1:59128 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:25:15.559839 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.559089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59142 (local address 127.9.91.125:36341)
0809 19:25:15.559120 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.559122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.559349 (+ 227us) server_negotiation.cc:407] Connection header received
0809 19:25:15.559435 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.559436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.559447 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.559472 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.559801 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59142: BlockingRecv error: recv got EOF from 127.0.0.1:59142 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 19:25:15.569666 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.568925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.9.91.125:36341)
0809 19:25:15.568956 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.568958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.569192 (+ 234us) server_negotiation.cc:407] Connection header received
0809 19:25:15.569279 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.569280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.569289 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.569315 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.569624 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59150: BlockingRecv error: recv got EOF from 127.0.0.1:59150 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 19:25:15.587915 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.587027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59156 (local address 127.9.91.125:36341)
0809 19:25:15.587061 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.587063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.587334 (+ 271us) server_negotiation.cc:407] Connection header received
0809 19:25:15.587429 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.587430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.587440 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.587484 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.587850 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59156: BlockingRecv error: recv got EOF from 127.0.0.1:59156 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 19:25:15.621867 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.621189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59172 (local address 127.9.91.125:36341)
0809 19:25:15.621233 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.621235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.621374 (+ 139us) server_negotiation.cc:407] Connection header received
0809 19:25:15.621423 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.621424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.621437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.621482 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.621808 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59172: BlockingRecv error: recv got EOF from 127.0.0.1:59172 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 19:25:15.687887 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.686972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.9.91.125:36341)
0809 19:25:15.687039 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.687042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.687178 (+ 136us) server_negotiation.cc:407] Connection header received
0809 19:25:15.687287 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.687288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.687298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.687336 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.687798 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59186: BlockingRecv error: recv got EOF from 127.0.0.1:59186 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 19:25:15.817705 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:15.817005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59196 (local address 127.9.91.125:36341)
0809 19:25:15.817079 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:15.817081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:15.817164 (+ 83us) server_negotiation.cc:407] Connection header received
0809 19:25:15.817240 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:15.817241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:15.817253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:15.817304 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:15.817642 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59196: BlockingRecv error: recv got EOF from 127.0.0.1:59196 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 19:25:16.075465 11842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 19:25:16.074874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59202 (local address 127.9.91.125:36341)
0809 19:25:16.074965 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:16.074968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:16.074980 (+ 12us) server_negotiation.cc:407] Connection header received
0809 19:25:16.075043 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:16.075044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:16.075054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:16.075104 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 19:25:16.075400 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59202: BlockingRecv error: recv got EOF from 127.0.0.1:59202 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 19:25:16.440537 11971 catalog_manager.cc:1259] Loaded cluster ID: 75ba29908e6149a391fa39ebaa212c0d
I20260809 19:25:16.440606 11971 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: loading cluster ID for follower catalog manager: success
I20260809 19:25:16.441357 11971 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: acquiring CA information for follower catalog manager: success
I20260809 19:25:16.441609 11971 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d91fbb1a9184fe3a05e016b6524b7ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:25:16.506398 12038 catalog_manager.cc:1259] Loaded cluster ID: 75ba29908e6149a391fa39ebaa212c0d
I20260809 19:25:16.506472 12038 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: loading cluster ID for follower catalog manager: success
I20260809 19:25:16.507251 12038 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: acquiring CA information for follower catalog manager: success
I20260809 19:25:16.507515 12038 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80dbcffec8864a99b6777511a7870118: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 19:25:16.591085 12144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:16.588484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.9.91.125:36341)
0809 19:25:16.588606 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:16.588608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:16.588690 (+ 82us) server_negotiation.cc:407] Connection header received
0809 19:25:16.588744 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:16.588745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:16.588814 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:16.588883 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:16.589030 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.589528 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.590287 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.590496 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.590551 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:16.590707 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:16.590827 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:16.590894 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:16.590973 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 19:25:16.590995 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260809 19:25:16.594800 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:16.592562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.9.91.124:41691)
0809 19:25:16.592669 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:16.592671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:16.592679 (+ 8us) server_negotiation.cc:407] Connection header received
0809 19:25:16.592761 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:16.592761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:16.592797 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:16.592862 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:16.592952 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.593422 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.594110 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.594309 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.594361 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:16.594498 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:16.594592 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:16.594653 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:16.594699 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 19:25:16.594721 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":57,"threads_started":1}
I20260809 19:25:16.598953 12144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:16.596468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.9.91.125:36341)
0809 19:25:16.596506 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:16.596508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:16.596644 (+ 136us) server_negotiation.cc:407] Connection header received
0809 19:25:16.596694 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:16.596695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:16.596703 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:16.596750 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:16.596839 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.597310 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.598343 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.598534 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.598584 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:16.598730 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:16.598761 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:16.598804 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:16.598845 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 19:25:16.598864 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 19:25:16.599115 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:16.596244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.9.91.124:41691)
0809 19:25:16.596281 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:16.596283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:16.596413 (+ 130us) server_negotiation.cc:407] Connection header received
0809 19:25:16.596513 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:16.596514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:16.596523 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:16.596573 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:16.596720 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.597398 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.598336 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.598608 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.598665 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:16.598777 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:16.598806 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:16.598848 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:16.599034 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 19:25:16.599056 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 19:25:16.599301 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 19:25:16.596189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.9.91.126:35749)
0809 19:25:16.596305 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 19:25:16.596307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 19:25:16.596533 (+ 226us) server_negotiation.cc:407] Connection header received
0809 19:25:16.596615 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 19:25:16.596616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 19:25:16.596666 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 19:25:16.596737 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 19:25:16.596905 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.597598 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.598295 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 19:25:16.598591 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 19:25:16.598648 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 19:25:16.598875 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 19:25:16.598995 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 19:25:16.599047 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 19:25:16.599190 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 19:25:16.599225 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260809 19:25:16.600821 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11639
I20260809 19:25:16.618635 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11734
I20260809 19:25:16.629142 9581 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskt8E36c/build/release/bin/kudu with pid 11848
2026-08-09T19:25:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2865 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5734 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (268021 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST